Erasure Code Calculator
Reference Hardware
Data Compression
Overview
MinIO Server supports compressing objects to reduce disk usage. Objects are compressed on PUT before writing to disk, and uncompressed on GET before they are sent to the client. This makes the compression process transparent to client applications and services.
Depending on the type of data, compression may also increase overall throughput. Write throughput for a production deployment is generally 500MB per second or greater per available CPU core in the system. Decompression is approximately 1 GB per second or greater for each CPU core.
For best results, review MinIO’s recommended hardware configuration or use MinIO SUBNET to work directly with engineers for analyzing compression performance.
Default File Types
Data compression is a global option, the configured settings apply to all buckets in a deployment. Enabling data compression compresses the following types of data by default:
File Extensions |
Media (MIME) Types |
|---|---|
|
|
You can control which objects are compressed by specifying the desired file extensions and media (MIME) types.
Existing objects are not modified
Enabling, disabling, or updating a deployment’s compression settings does not modify existing objects. New objects are compressed according to the settings in effect at the time they are created.
Excluded File Types
Some data cannot be effectively compressed, such as video or already compressed data. MinIO does not compress common incompressible file types, even if they are specified in the compression configuration.
Objects of these types are never compressed:
Object Type |
File Extension |
Media (MIME) Type |
|---|---|---|
Audio |
|
|
Video |
*.mp4*.mkv*.mov |
|
Image |
*.jpg*.png*.gif |
|
7ZIP Compressed |
|
|
BZIP2 Compressed |
|
|
GZIP Compressed |
|
|
RAR Compressed |
|
|
LZMA Compressed |
|
|
ZIP Compressed |
|
application/zipapplication-x-zip-compressed |
Data Compression and Encryption
MinIO supports encrypting compressed objects but recommends against combining compression and encryption without a prior risk assessment. Before enabling encryption for compressed objects, carefully consider the security needs of your environment.
See Transparent Data Compression on MinIO for more about combining compression and encryption. MinIO SUBNET users can log in and engage with our engineering and security teams to review encryption options.
Tutorials
Enable Data Compression
To enable data compression, use mc admin config set to set the compression key enable option to on.
The following enables compression for new objects of the default types:
mc admin config set ALIAS compression enable=on
Replace
ALIASwith thealiasof a configured MinIO deployment.
Existing uncompressed objects are not modified. To configure which extensions and types to compress, see Configure Which Objects to Compress.
To view the current compression settings:
mc admin config set ALIAS compression
Disable Data Compression
To disable data compression, use mc admin config set to set the compression key enable option to off:
The following disables data compression for new objects:
mc admin config set ALIAS compression enable=off
Replace
ALIASwith thealiasof a configured MinIO deployment.
Existing compressed objects are not modified.
Configure Which Objects to Compress
Configure the objects to compress by specifying the desired file extensions and media types in extensions or mime_types arguments.
The default data compression configuration compresses the following types of data:
File Extensions |
Media (MIME) Types |
|---|---|
|
|
Default excluded extensions and types are never compressed
Some objects cannot be efficiently compressed.
MinIO will not attempt to compress these objects, even if they are specified in extensions or mime_types arguments.
See Excluded File Types for a list of excluded types.
The sections below describe how to configure compression for the desired file extensions and media types.
Compress All Compressible Objects
To compress all objects except the default excluded types, use mc admin config set to set the compression key extensions and mime_types options to empty lists:
mc admin config set ALIAS compression extensions= mime_types=
Replace
ALIASwith thealiasof a configured MinIO deployment.
Compress Objects by File Extension
To compress objects with certain file extensions, use mc admin config set to set the desired file extensions in an extensions argument.
The following command compresses files with the extensions .bin and .txt:
mc admin config set ALIAS compression extensions=".bin, .txt"
Replace
ALIASwith thealiasof a configured MinIO deployment.
The new list of file extensions replaces the previous list.
To add or remove an extension, repeat the extensions command with the complete list of extensions to compress.
The following adds .pdf to the list of file extensions from the previous example:
mc admin config set ALIAS compression extensions=".bin, .txt, .pdf"
Replace
ALIASwith thealiasof a configured MinIO deployment.
Compress Objects by Media Type
To compress objects of certain media types, use mc admin config set to set the compression key mime_types option to a list of the desired types.
The following example compresses files of types application/json and image/bmp:
mc admin config set ALIAS compression mime_types="application/json, image/bmp"
Replace
ALIASwith thealiasof a configured MinIO deployment.
The new list of media types replaces the previous list.
To add or remove a type, repeat the mime_types command with the complete list of types to compress.
You can use * to specify all subtypes of a single media type.
The following command adds all text subtypes to the list from the previous example:
mc admin config set ALIAS compression mime_types="application/json, image/bmp, text/*"
Replace
ALIASwith thealiasof a configured MinIO deployment.
