Server-Side Object Encryption with Hashicorp Vault Root KMS
MinIO Server-Side Encryption (SSE) protects objects as part of write operations, allowing clients to take advantage of server processing power to secure objects at the storage layer (encryption-at-rest). SSE also provides key functionality to regulatory and compliance requirements around secure locking and erasure.
MinIO SSE uses Key Encryption Service (KES) and an external root Key Management Service (KMS) for performing secured cryptographic operations at scale. The root KMS provides stateful and secured storage of External Keys (EK) while KES is stateless and derives additional cryptographic keys from the root-managed EK.
This procedure provides guidance for deploying and configuring KES at scale for a supporting SSE on a production MinIO deployment. You can also use this procedure for deploying to local environments for testing and evaluation.
As part of this procedure, you will:
Deploy one or more KES servers configured to use Hashicorp Vault as the root KMS. You may optionally deploy a load balancer for managing connections to those KES servers.
Create a new EK on Vault for use with SSE.
Create or modify a MinIO deployment with support for SSE using KES. Defer to the Deploy Distributed MinIO tutorial for guidance on production-ready MinIO deployments.
Configure automatic bucket-default SSE-KMS
For production orchestrated environments, use the MinIO Kubernetes Operator to deploy a tenant with SSE enabled and configured for use with Hashicorp Vault.
Important
Enabling SSE on a MinIO deployment automatically encrypts the backend data for that deployment using the default encryption key.
MinIO requires access to KES and the root KMS to decrypt the backend and start normally. You cannot disable KES later or “undo” the SSE configuration at a later point.
Prerequisites
Deploy or Ensure Access to a Hashicorp Vault Service
This procedure assumes an existing Hashicorp Vault installation accessible from the local host. The Vault Quick Start provides a sufficient foundation for the purposes of this procedure. Defer to the Vault Documentation for guidance on deployment and configuration.
KES Operations Require Unsealed Vault
You must unseal the Vault instance to allow any cryptographic operations, including key creation and retrieval. KES returns an error if the configured Vault service is sealed.
If you restart or otherwise seal the Vault instance, KES cannot perform any cryptographic operations against the Vault. You must unseal the Vault to ensure normal operations.
See the Vault documentation on Seal/Unseal for more information.
MinIO KES supports either the V1 or V2 Vault K/V engines.
MinIO KES requires using AppRole authentication to the Vault server. You must create an AppRole, assign it a policy that the necessary permissions, and retrieve the AppRole ID and Secret for use in configuring KES.
You can use the following steps to enable AppRole authentication and create the necessary policies to support core KES functionality against Vault:
Enable AppRole Authentication
vault auth enable approle
Create a Policy for KES
Create a policy with necessary capabilities for KES to use when accessing Vault. Select the tab corresponding to the KV engine used for storing KES secrets:
Create an access policy
kes-policy.hcl
with a configuration similar to the following:path "kv/*" { capabilities = [ "create", "read", "delete" ] }
Write the policy to Vault using
vault policy write kes-policy kes-policy.hcl
.Create an access policy
kes-policy.hcl
with a configuration similar to the following:path "kv/data/*" { capabilities = [ "create", "read"] } path "kv/metadata/*" { capabilities = [ "list", "delete"] }
Write the policy to Vault using
vault policy write kes-policy kes-policy.hcl
Create an AppRole for KES and assign it the created policy
vault write auth/approle/role/kes-role token_num_uses=0 secret_id_num_uses=0 period=5m vault write auth/approle/role/kes-role policies=kes-policy
Retrieve the AppRole ID and Secret
vault read auth/approle/role/kes-role/role-id vault write -f auth/approle/role/kes-role/secret-id
Deploy or Ensure Access to a MinIO Deployment
This procedure provides instructions for modifying the startup environment variables of a MinIO deployment to enable SSE via KES and the root KMS.
For instructions on new production deployments, see the Multi-Node Multi-Drive (Distributed) tutorial. For instructions on new local or evaluation deployments, see the Single-Node Single-Drive tutorial.
When creating the environment file for the deployment, pause and switch back to this tutorial to include the necessary environment variables to support SSE.
For existing MinIO Deployments, you can modify the existing environment file and restart the deployment as instructed during this procedure.
Enable Server-Side Encryption using Hashicorp Vault for Local Development
This procedure assumes deploying MinIO and KES onto the same host. You can use and modify the information below to better suit your local development environment.
This procedure assumes the following:
An existing Vault deployment
A single host machine for deploying KES and MinIO
Set Up Folder Hierarchy
Create the following folders on the host machine if they do not already exist.
mkdir -P /opt/kes/certs
mkdir -P /opt/kes/config
mkdir -P /opt/minio/certs
mkdir -P /opt/minio/config
mkdir -P ~/minio
This procedure uses these paths in the following steps. If you use different paths, make the necessary modifications for each step to reflect those changes
1) Download the KES Server Binary
Download the latest stable release (2023-05-02T22-48-10Z) of KES from github.com/minio/kes.
Select the binary appropriate for the host OS architecture.
For example, hosts running X86-64 (Intel/AMD64) should download the kes-linux-amd64
package.
The following example code downloads the latest Linux AMD64-compatible binary and moves it to the system PATH
:
curl --retry 10 https://github.com/minio/kes/releases/download/2023-05-02T22-48-10Z/kes-linux-amd64 -o /tmp/kes
chmod +x /tmp/kes
sudo mv /tmp/kes /usr/local/bin
kes --version
For distributed KES topologies, repeat this step and all following KES-specific instructions for each host on which you want to deploy KES. MinIO uses a round-robin approach by default for routing connections to multiple configured KES servers. For more granular controls, deploy a dedicated load balancer to manage connections to distributed KES hosts.
2) Generate TLS Certificates for KES and MinIO
The following commands create two TLS certificates that expire within 30 days of creation:
A TLS certificate for KES to secure communications between it and the Vault deployment
A TLS certificate for MinIO to perform mTLS authentication to KES.
Use Caution in Production Environments
DO NOT use the TLS certificates generated as part of this procedure for any long-term development or production environments.
Defer to organization/industry best practices around TLS certificate generation and management. A complete guide to creating valid certificates (e.g. well-formed, current, and trusted) is beyond the scope of this procedure.
# These commands output keys to /opt/kes/certs
# and /opt/minio/certs respectively
kes identity new kes_server \
--key /opt/kes/certs/kes-server.key \
--cert /opt/kes/certs/kes-server.cert \
--ip "127.0.0.1" \
--dns localhost
kes identity new minio_server \
--key /opt/minio/certs/minio-kes.key \
--cert /opt/minio/certs/minio-kes.cert \
--ip "127.0.0.1" \
--dns localhost
The --ip
and --dns
parameters set the IP and DNS SubjectAlternativeName
for the certificate.
The above example assumes that all components (Vault, MinIO, and KES) deploy on the same local host machine accessible via localhost
or 127.0.0.1
.
You can specify additional IP or Hostnames based on the network configuration of your local host.
Depending on your Vault configuration, you may need to pass the kes-server.cert
certificate as a trusted Certificate Authority.
See the Hashicorp Server Configuration Documentation for more information.
Defer to the client documentation for instructions on trusting a third-party CA.
3) Create the KES and MinIO Configurations
Create the KES Configuration File
Create the configuration file using your preferred text editor. The following example uses
nano
:nano /opt/kes/config/kes-config.yaml
KES uses a YAML-formatted configuration file. The following YAML provides the minimum required fields for using Hashicorp Vault as the root KMS. You must modify this YAML to reflect your deployment environment.
address: 0.0.0.0:7373 # Disable the root administrator identity, as we do not need that level of access for # supporting SSE operations. admin: identity: disabled # Specify the TLS keys generated in the previous step here # For production environments, use keys signed by a known and trusted # Certificate Authority (CA). tls: key: /opt/kes/certs/kes-server.key cert: /opt/kes/certs/kes-server.cert # Sets access policies for KES # The `minio` policy grants access to the listed APIs. policy: minio: allow: - /v1/key/create/* # You can replace these wildcard '*' with a string prefix to restrict key names - /v1/key/generate/* # e.g. '/minio-' - /v1/key/decrypt/* - /v1/key/bulk/decrypt - /v1/key/list - /v1/status - /v1/metrics - /v1/log/audit - /v1/log/error identities: - MINIO_IDENTITY_HASH # Replace with the output of 'kes identity of minio-kes.cert' # In production environments, each client connecting to KES must # Have their TLS hash listed under at least one `policy`. # Specify the connection information for the Vault server. # The endpoint should be resolvable from the host. # This example assumes that Vault is configured with an AppRole ID and # Secret for use with KES. keystore: vault: endpoint: https://HOSTNAME:8200 engine: "/path/to/engine" # Replace with the path to the K/V Engine version: "v1|v2" # Specify v1 or v2 depending on the version of the K/V Engine approle: id: "VAULTAPPID" # Hashicorp Vault AppRole ID secret: "VAULTAPPSECRET" # Hashicorp Vault AppRole Secret ID retry: 15s status: ping: 10s # Required if Vault uses certificates signed by an unknown CA, # e.g. self-signed or internal (non-globally trusted). # Replace this value with the full path to the Vault CA certificate. tls: ca: vault-tls-CA.cert
Set
MINIO_IDENTITY_HASH
to the identity hash of the MinIO mTLS certificate.The following command computes the necessary hash:
kes identity of /opt/minio/certs/minio-kes.cert
Replace the
vault.endpoint
with the hostname of the Vault server(s).Set the
vault.engine
andvault.version
to the appropriate values for the Vault K/V Engine configurationReplace the
VAULTAPPID
andVAULTAPPSECRET
with the appropriate Vault AppRole credentials.
Create the MinIO Environment File
Create or modify the environment file for the MinIO deployment using your preferred text editor. The following example uses
nano
:nano /opt/minio/config/minio
Add the following lines to the MinIO Environment file on each MinIO host. See the tutorials for Deploy MinIO: Single-Node Single-Drive, Deploy MinIO: Single-Node Multi-Drive, or Deploy MinIO: Multi-Node Multi-Drive for more detailed descriptions of a base MinIO environment file.
# Add these environment variables to the existing environment file MINIO_KMS_KES_ENDPOINT=https://HOSTNAME:7373 MINIO_KMS_KES_CERT_FILE=/opt/minio/certs/minio-kes.cert MINIO_KMS_KES_KEY_FILE=/opt/minio/certs/minio-kes.key # Allows validation of the KES Server Certificate (Self-Signed or Third-Party CA) # Change this path to the location of the KES CA Path MINIO_KMS_KES_CAPATH=/opt/kes/certs/kes-server.cert # Sets the default KMS key for the backend and SSE-KMS/SSE-S3 Operations) MINIO_KMS_KES_KEY_NAME=minio-backend-default-key # Optional, defines the name for the KES server enclave to use. MINIO_KMS_KES_ENCLAVE=<name>
Replace
HOSTNAME
with the IP address or hostname of the KES server. If the MinIO server host machines cannot resolve or reach the specifiedHOSTNAME
, the deployment may return errors or fail to start.If using a single KES server host, specify the IP or hostname of that host
If using multiple KES server hosts, specify a comma-separated list of IPs or hostnames of each host
MinIO uses the
MINIO_KMS_KES_KEY_NAME
key for the following cryptographic operations:Encrypting the MinIO backend (IAM, configuration, etc.)
Encrypting objects using SSE-KMS if the request does not include a specific EK.
Encrypting objects using SSE-S3.
MinIO uses the
MINIO_KMS_KES_ENCLAVE
key to define the name of the KES enclave to use.Replace
<name>
with the name of the enclave to use.If not defined, MinIO does not send any enclave information. This may result in using the default enclave for stateful KES servers.
A KES enclave provides an isolated space for its associated keys separate from other enclaves on a stateful KES server.
The
minio-kes
certificates enable mTLS between the MinIO deployment and the KES server only. They do not otherwise enable TLS for other client connections to MinIO.
4) Start KES and MinIO
KES Operations Requires Unsealed Vault
You must unseal the Vault instance to allow normal cryptographic operations, including key creation or retrieval. See the Vault documentation on Seal/Unseal for more information.
You must start KES before starting MinIO. The MinIO deployment requires access to KES as part of its startup.
Start the KES Server
Run the following commands in a terminal or shell to start the KES server as a foreground process:
sudo setcap cap_ipc_lock=+ep $(readlink -f $(which kes)) kes server --auth=off --config=/opt/kes/config/kes-config.yaml
The first command allows KES to use the mlock system call without running as root.
mlock
ensures the OS does not write in-memory data to a drive (swap memory) and mitigates the risk of cryptographic operations being written to unsecured drive at any time. KES 0.21.0 and later automatically detect and enablemlock
if supported by the host OS. Versions 0.20.0 and earlier required specifying the--mlock
argument to KES.The second command starts the KES server in the foreground using the configuration file created in the last step. The
--auth=off
disables strict validation of client TLS certificates. Using self-signed certificates for either the MinIO client or the root KMS server requires specifying this option.KES listens on port
7373
by default. You can monitor the server logs from the terminal session. If you run KES without tying it to the current shell session (e.g. withnohup
), use that method’s associated logging system (e.g.nohup.txt
).Start the MinIO Server
Run the following command in a terminal or shell to start the MinIO server as a foreground process.
export MINIO_CONFIG_ENV_FILE=/opt/minio/config/minio minio server --console-address :9090
Foreground processes depend on the shell or terminal in which they run. Exiting or terminating the shell/terminal instance also kills the attached process. Defer to your operating system best practices for running processes in the background.
5) Generate a New Encryption Key
KES Operations Requires Unsealed Vault
You must unseal the Vault instance to allow normal cryptographic operations, including key creation or retrieval. See the Vault documentation on Seal/Unseal for more information.
MinIO requires that the EK exist on the root KMS before performing SSE operations using that key.
Use kes key create
or mc admin kms key create
to add a new EK for use with SSE.
The following command uses the kes key create
command to add a new External Key (EK) stored on the root KMS server for use with encrypting the MinIO backend.
export KES_SERVER=https://127.0.0.1:7373
export KES_CLIENT_KEY=/opt/minio/certs/minio-kes.key
export KES_CLIENT_CERT=/opt/minio/certs/minio-kes.cert
kes key create -k encrypted-bucket-key
6) Enable SSE-KMS for a Bucket
You can use either the MinIO Console or the MinIO mc
CLI to enable bucket-default SSE-KMS with the generated key:
Open the MinIO Console by navigating to http://127.0.0.1:9090 in your preferred browser and logging in with the root credentials specified to the MinIO container.
If you deployed MinIO using a different Console listen port, substitute 9090
with that port value.
Once logged in, create a new Bucket and name it to your preference. Select the Gear icon to open the management view.
Select the pencil icon next to the Encryption field to open the modal for configuring a bucket default SSE scheme.
Select SSE-KMS, then enter the name of the key created in the previous step.
Once you save your changes, try to upload a file to the bucket. When viewing that file in the object browser, note that in the sidebar the metadata includes the SSE encryption scheme and information on the key used to encrypt that object. This indicates the successful encrypted state of the object.
The following commands:
Create a new alias for the MinIO deployment
Create a new bucket for storing encrypted data
Enable SSE-KMS encryption on that bucket
mc alias set local http://127.0.0.1:9000 ROOTUSER ROOTPASSWORD
mc mb local/encryptedbucket
mc encrypt set SSE-KMS encrypted-bucket-key ALIAS/encryptedbucket
Write a file to the bucket using mc cp
or any S3-compatible SDK with a PutObject
function.
You can then run mc stat
on the file to confirm the associated encryption metadata.
Enable Server-Side Encryption using Hashicorp Vault in Production Environments
This procedure provides instructions for configuring and enabling Server-Side Encryption using Hashicorp Vault in production environments. Specifically, this procedure assumes the following:
An existing production-grade Vault deployment
One or more hosts for deploying KES
One or more hosts for a new or existing MinIO deployment
Set Up Folder Hierarchy
Create the following folders on the KES and MinIO host machines if they do not already exist:
mkdir -P /opt/kes/certs
mkdir -P /opt/kes/config
mkdir -P /opt/minio/certs
This procedure uses these paths in the following steps. If you use different paths, make the necessary modifications for each step to reflect those changes
1) Download KES and Create the Service File
Download KES
Download the latest stable release (2023-05-02T22-48-10Z) of KES from github.com/minio/kes.
Select the binary appropriate for the host OS architecture. For example, hosts running X86-64 (Intel/AMD64) should download the
kes-linux-amd64
package.The following example code downloads the latest Linux AMD64-compatible binary and moves it to the system
PATH
:curl --retry 10 https://github.com/minio/kes/releases/download/2023-05-02T22-48-10Z/kes-linux-amd64 -o /tmp/kes chmod +x /tmp/kes sudo mv /tmp/kes /usr/local/bin kes --version
For distributed KES topologies, repeat this step and all following KES-specific instructions for each host on which you want to deploy KES. MinIO uses a round-robin approach by default for routing connections to multiple configured KES servers. For more granular controls, deploy a dedicated load balancer to manage connections to distributed KES hosts.
Create the Service File
Create the
/etc/systemd/system/kes.service
file on all KES hosts:[Unit] Description=KES Documentation=https://github.com/minio/kes/wiki Wants=network-online.target After=network-online.target AssertFileIsExecutable=/usr/local/bin/kes [Service] WorkingDirectory=/etc/kes/ AmbientCapabilities=CAP_IPC_LOCK User=kes Group=kes ProtectProc=invisible ExecStart=/usr/local/bin/kes server --config=/opt/kes/config.yaml # Let systemd restart this service always Restart=always # Specifies the maximum file descriptor number that can be opened by this process LimitNOFILE=65536 # Specifies the maximum number of threads this process can create TasksMax=infinity # Disable timeout logic and wait until process is stopped TimeoutStopSec=infinity SendSIGKILL=no [Install] WantedBy=multi-user.target
You may need to run
systemctl daemon-reload
to load the new service file intosystemctl
.The
kes.service
file runs as thekes
User and Group by default. You can create the user and group using theuseradd
andgroupadd
commands. The following example creates the user and group. These commands typically require root (sudo
) permissions.groupadd -r kes useradd -M -r -g kes kes
The
kes
user and group must have read access to all files used by the KES service:chown -R kes:kes /opt/kes
2) Generate TLS Certificates for KES and MinIO
Enabling connectivity between MinIO and KES requires at minimum one TLS certificate for performing mutual TLS (mTLS) authentication. Depending on your Vault configuration, you may also need to create a dedicated set of TLS certificates for KES to connect and authenticate to Vault. Defer to your organizations best practices around generating production-ready TLS certificates.
Place the certificates and corresponding private keys an appropriate directory such that the MinIO and KES service users can access and read their contents. The following example structure uses the folder hierarchy suggested in the beginning of this procedure:
-rw-r--r-- 1 kes:kes /opt/kes/certs/kes-server.cert
-rw-r--r-- 1 kes:kes /opt/kes/certs/kes-server.key
# If the Vault certs are self-signed or use a non-global CA
# Include those CA certs as well
-rw-r--r-- 1 kes:kes /opt/kes/certs/vault-CA.cert
-rw-r--r-- 1 minio-user:minio-user /opt/minio/certs/minio-kes.cert
-rw-r--r-- 1 minio-user:minio-user /opt/minio/certs/minio-kes.key
# If KES certs are self-signed or use a non-global CA
# Include the CA certs as well
-rw-r--r-- 1 minio-user:minio-user /opt/minio/certs/kes-server.cert
The general strategy for cert management is to ensure that each process (MinIO, KES, and Vault) have their own mTLS certificates and the Certificate Authority (CA) used to sign each client certificate.
Depending on your Vault configuration, you may also need to specify the CA used to sign the KES certificates to the Vault server. See the Hashicorp Vault Configuration Docs for more information. Defer to the client documentation for instructions on trusting a third-party CA.
3) Create the KES and MinIO Configurations
Important
Starting with https://github.com/minio/minio/releases/tag/RELEASE.2023-02-17T17-52-43Z, MinIO requires expanded KES permissions for functionality. The example configuration in this section contains all required permissions.
Create the KES Configuration File
Create the configuration file using your preferred text editor. The following example uses
nano
:nano /opt/kes/config.yaml
KES uses a YAML-formatted configuration file. The following YAML provides the minimum required fields for using Hashicorp Vault as the root KMS. You must modify this YAML to reflect your deployment environment.
address: 0.0.0.0:7373 # Disable the root administrator identity, as we do not need that level of access for # supporting SSE operations. admin: identity: disabled # Specify the TLS keys generated in the previous step here # For production environments, use keys signed by a known and trusted # Certificate Authority (CA). tls: key: /opt/kes/certs/kes-server.key cert: /opt/kes/certs/kes-server.cert # Sets access policies for KES # The `minio` policy grants access to the listed APIs. policy: minio: allow: - /v1/key/create/* # You can replace these wildcard '*' with a string prefix to restrict key names - /v1/key/generate/* # e.g. '/minio-' - /v1/key/decrypt/* - /v1/key/bulk/decrypt - /v1/key/list - /v1/status - /v1/metrics - /v1/log/audit - /v1/log/error identities: - MINIO_IDENTITY_HASH # Replace with the output of 'kes identity of minio-kes.cert' # In production environments, each client connecting to KES must # Have their TLS hash listed under at least one `policy`. # Specify the connection information for the Vault server. # The endpoint should be resolvable from the host. # This example assumes that Vault is configured with an AppRole ID and # Secret for use with KES. keystore: vault: endpoint: https://HOSTNAME:8200 engine: "/path/to/engine" # Replace with the path to the K/V Engine version: "v1|v2" # Specify v1 or v2 depending on the version of the K/V Engine approle: id: "VAULTAPPID" # Hashicorp Vault AppRole ID secret: "VAULTAPPSECRET" # Hashicorp Vault AppRole Secret ID retry: 15s status: ping: 10s # Required if Vault uses certificates signed by an unknown CA, # e.g. self-signed or internal (non-globally trusted). # Replace this value with the full path to the Vault CA certificate. tls: ca: vault-tls-CA.cert
Set
MINIO_IDENTITY_HASH
to the identity hash of the MinIO mTLS certificate.The following command computes the necessary hash:
kes identity of /opt/minio/certs/minio-kes.cert
Replace the
keystore.vault.endpoint
with the hostname of the Vault server(s).Replace
keystore.vault.engine
andkeystore.vault.version
with the path and version of the KV engine used for storing secrets.Replace the
VAULTAPPID
andVAULTAPPSECRET
with the appropriate Vault AppRole credentials.Modify the
keystore.vault.tls.ca
value to correspond to the path to the Vault CA certificate used to sign the Vault TLS keys.
Configure the MinIO Environment File
Modify the MinIO Server environment file for all hosts in the target deployment to include the following environment variables.
MinIO defaults to expecting this file at
/etc/default/minio
. If you modified your deployment to use a different location for the environment file, modify the file at that location.Add the following lines to the MinIO Environment file on each MinIO host. See the tutorials for Deploy MinIO: Single-Node Single-Drive, Deploy MinIO: Single-Node Multi-Drive, or Deploy MinIO: Multi-Node Multi-Drive for more detailed descriptions of a base MinIO environment file.
# Add these environment variables to the existing environment file MINIO_KMS_KES_ENDPOINT=https://HOSTNAME:7373 MINIO_KMS_KES_CERT_FILE=/opt/minio/certs/minio-kes.cert MINIO_KMS_KES_KEY_FILE=/opt/minio/certs/minio-kes.key # Allows validation of the KES Server Certificate (Self-Signed or Third-Party CA) # Change this path to the location of the KES CA Path MINIO_KMS_KES_CAPATH=/opt/kes/certs/kes-server.cert # Sets the default KMS key for the backend and SSE-KMS/SSE-S3 Operations) MINIO_KMS_KES_KEY_NAME=minio-backend-default-key # Optional, defines the name for the KES server enclave to use. MINIO_KMS_KES_ENCLAVE=<name>
Replace
HOSTNAME
with the IP address or hostname of the KES server. If the MinIO server host machines cannot resolve or reach the specifiedHOSTNAME
, the deployment may return errors or fail to start.If using a single KES server host, specify the IP or hostname of that host
If using multiple KES server hosts, specify a comma-separated list of IPs or hostnames of each host
MinIO uses the
MINIO_KMS_KES_KEY_NAME
key for the following cryptographic operations:Encrypting the MinIO backend (IAM, configuration, etc.)
Encrypting objects using SSE-KMS if the request does not include a specific EK.
Encrypting objects using SSE-S3.
MinIO uses the
MINIO_KMS_KES_ENCLAVE
key to define the name of the KES enclave to use.Replace
<name>
with the name of the enclave to use.If not defined, MinIO does not send any enclave information. This may result in using the default enclave for stateful KES servers.
A KES enclave provides an isolated space for its associated keys separate from other enclaves on a stateful KES server.
The
minio-kes
certificates enable mTLS between the MinIO deployment and the KES server only. They do not otherwise enable TLS for other client connections to MinIO.
4) Start KES and MinIO
KES Operations Requires Unsealed Vault
You must unseal the Vault instance to allow normal cryptographic operations, including key creation or retrieval. See the Vault documentation on Seal/Unseal for more information.
You must start KES before starting MinIO. The MinIO deployment requires access to KES as part of its startup.
This step uses systemd
for starting and managing both the KES and MinIO server processes:
Start the KES Service on All Hosts
Run the following command on each KES host to start the service:
systemctl start kes
You can validate the startup by using
systemctl status kes
. If the service started successfully, usejournalctl -uf kes
to check the KES output logs.Start the MinIO Server
For new MinIO deployments, run the following command on each MinIO host to start the service:
systemctl start minio
For existing MinIO deployments, run the following command on each MinIO host to restart the service:
systemctl reload minio systemctl restart minio
5) Generate a New Encryption Key
KES Operations Requires Unsealed Vault
You must unseal the Vault instance to allow normal cryptographic operations, including key creation or retrieval. See the Vault documentation on Seal/Unseal for more information.
MinIO requires that the EK exist on the root KMS before performing SSE operations using that key.
Use kes key create
or mc admin kms key create
to add a new EK for use with SSE.
The following command uses the kes key create
command to add a new External Key (EK) stored on the root KMS server for use with encrypting the MinIO backend.
export KES_SERVER=https://127.0.0.1:7373
export KES_CLIENT_KEY=/opt/minio/certs/minio-kes.key
export KES_CLIENT_CERT=/opt/minio/certs/minio-kes.cert
kes key create -k encrypted-bucket-key
6) Enable SSE-KMS for a Bucket
You can use either the MinIO Console or the MinIO mc
CLI to enable bucket-default SSE-KMS with the generated key:
Open the MinIO Console by navigating to http://127.0.0.1:9090 in your preferred browser and logging in with the root credentials specified to the MinIO container.
If you deployed MinIO using a different Console listen port, substitute 9090
with that port value.
Once logged in, create a new Bucket and name it to your preference. Select the Gear icon to open the management view.
Select the pencil icon next to the Encryption field to open the modal for configuring a bucket default SSE scheme.
Select SSE-KMS, then enter the name of the key created in the previous step.
Once you save your changes, try to upload a file to the bucket. When viewing that file in the object browser, note that in the sidebar the metadata includes the SSE encryption scheme and information on the key used to encrypt that object. This indicates the successful encrypted state of the object.
The following commands:
Create a new alias for the MinIO deployment
Create a new bucket for storing encrypted data
Enable SSE-KMS encryption on that bucket
mc alias set local http://127.0.0.1:9000 ROOTUSER ROOTPASSWORD
mc mb local/encryptedbucket
mc encrypt set SSE-KMS encrypted-bucket-key ALIAS/encryptedbucket
Write a file to the bucket using mc cp
or any S3-compatible SDK with a PutObject
function.
You can then run mc stat
on the file to confirm the associated encryption metadata.
Configuration Reference for Hashicorp Vault
The following section describes each of the Key Encryption Service (KES) configuration settings for using Hashicorp Vault as the root Key Management Service (KMS) for SSE.
Important
Starting with https://github.com/minio/minio/releases/tag/RELEASE.2023-02-17T17-52-43Z, MinIO requires expanded KES permissions for functionality. The example configuration in this section contains all required permissions.
The following YAML describes the minimum required fields for configuring Hashicorp Vault as an external KMS for supporting SSE.
Fields with ${<STRING>}
use the environment variable matching the <STRING>
value.
You can use this functionality to set credentials without writing them to the configuration file.
The YAML assumes a minimal set of permissions for the MinIO deployment accessing KES.
As an alternative, you can omit the policy.minio-server
section and instead set the ${MINIO_IDENTITY}
hash as the ${ROOT_IDENTITY}
.
address: 0.0.0.0:7373
root: ${ROOT_IDENTITY}
tls:
key: kes-server.key
cert: kes-server.cert
policy:
minio-server:
allow:
- /v1/key/create/*
- /v1/key/generate/*
- /v1/key/decrypt/*
- /v1/key/bulk/decrypt
- /v1/key/list
- /v1/status
- /v1/metrics
- /v1/log/audit
- /v1/log/error
identities:
- ${MINIO_IDENTITY}
keys:
- name: "minio-encryption-key-alpha"
- name: "minio-encryption-key-baker"
- name: "minio-encryption-key-charlie"
keystore:
vault:
endpoint: https://vault.example.net:8200
engine: "kv"
version: "v1"
namespace: "minio"
prefix: "keys"
approle:
id: ${KES_APPROLE_ID}
secret: ${KES_APPROLE_SECRET}
retry: 15s
status:
ping: 10s
tls:
key: "kes-mtls.key"
cert: "kes-mtls.cert"
ca: vault-tls.cert
Key |
Description |
---|---|
|
The network address and port the KES server listens to on startup.
Defaults to port |
|
The identity for the KES superuser ( Specify |
|
The TLS private key and certificate used by KES for establishing TLS-secured communications.
Specify the full path for both the private |
|
Specify one or more policies to control access to the KES server. MinIO SSE requires access to the following KES cryptographic APIs:
Specifying additional keys does not expand MinIO SSE functionality and may violate security best practices around providing unnecessary client access to cryptographic key operations. You can restrict the range of key names MinIO can create as part of performing
SSE by specifying a prefix before the KES uses mTLS to authorize connecting clients by comparing the
hash of the TLS certificate against the |
|
Specify an array of keys which must exist on the root KMS for KES to successfully start. KES attempts to create the keys if they do not exist and exits with an error if it fails to create any key. KES does not accept any client requests until it completes validation of all specified keys. |
|
The configuration for the Hashicorp Vault keystore. The following fields are required:
|