Server-Side Object Encryption with Hashicorp Vault Root KMS

Deploy or Ensure Access to a Hashicorp Vault Service

This procedure assumes an existing Hashicorp Vault installation accessible from the local host. The Vault Quick Start provides a sufficient foundation for the purposes of this procedure. Defer to the Vault Documentation for guidance on deployment and configuration.

MinIO KES supports either the V1 or V2 Vault K/V engines.

MinIO KES requires using AppRole authentication to the Vault server. You must create an AppRole, assign it a policy that the necessary permissions, and retrieve the AppRole ID and Secret for use in configuring KES.

You can use the following steps to enable AppRole authentication and create the necessary policies to support core KES functionality against Vault:

1. Enable AppRole Authentication

   vault auth enable approle
   

2. Create a Policy for KES

   Create a policy with necessary capabilities for KES to use when accessing Vault. Select the tab corresponding to the KV engine used for storing KES secrets:

   Vault Engine V1

   Create an access policy kes-policy.hcl with a configuration similar to the following:

   path "kv/*" {
         capabilities = [ "create", "read", "delete" ]
   }
   

   Write the policy to Vault using vault policy write kes-policy kes-policy.hcl.

   Vault Engine V2

   Create an access policy kes-policy.hcl with a configuration similar to the following:

   path "kv/data/*" {
         capabilities = [ "create", "read"]
   }
   
   path "kv/metadata/*" {
         capabilities = [ "list", "delete"]
   }
   

   Write the policy to Vault using vault policy write kes-policy kes-policy.hcl

3. Create an AppRole for KES and assign it the created policy

   vault write    auth/approle/role/kes-role token_num_uses=0 secret_id_num_uses=0 period=5m
   vault write    auth/approle/role/kes-role policies=kes-policy
   

4. Retrieve the AppRole ID and Secret

   vault read     auth/approle/role/kes-role/role-id
   vault write -f auth/approle/role/kes-role/secret-id
   

Install Podman or a Similar Container Management Interface

This procedure assumes you have a working Podman installation configured to run in “Rootfull” mode.

“Rootless” modes may not provide sufficient permissions to run KES with the necessary security settings. See the relevant “rootless” documentation for more information.

1) Generate TLS Certificates for KES and MinIO

The following commands create two TLS certificates that expire within 30 days of creation:

• A TLS certificate for KES to secure communications between it and the Hashicorp Vault service.

• A TLS certificate for MinIO to perform mTLS authentication to KES.

# These commands output keys to ~/minio-kes-vault/certs and ~/minio-kes-vault/certs on the host operating system

podman run --rm  \
  -v ~/minio-kes-vault/certs:/certs  \
  quay.io/minio/kes:2024-01-11T13-09-29Z identity new  kes_server \
    --key  /certs/kes-server.key  \
    --cert /certs/kes-server.cert  \
    kes-server

podman run --rm  \
  -v ~/minio-kes-vault/certs:/certs  \
  quay.io/minio/kes:2024-01-11T13-09-29Z identity new minio_server \
    --key  /certs/minio-kes.key  \
    --cert /certs/minio-kes.cert  \
    minio-server

Depending on your Vault configuration, you may need to pass the kes-server.cert as a trusted Certificate Authority. See the Hashicorp Vault Configuration Docs for more information. Defer to the client documentation for instructions on trusting a third-party CA.

2) Create the KES and MinIO Configurations

1. Create the KES Configuration File

   Create the configuration file using your preferred text editor. The following example uses nano:

   nano ~/minio-kes-vault/config/kes-config.yaml
   

   KES uses a YAML-formatted configuration file. The following YAML provides the minimum required fields for using Hashicorp Vault as the root KMS. You must modify this YAML to reflect your deployment environment.

   address: 0.0.0.0:7373
   
   # Disable the root administrator identity, as we do not need that level of access for
   # supporting SSE operations.
   admin:
     identity: disabled
   
   # Specify the TLS keys generated in the previous step here
   # For production environments, use keys signed by a known and trusted Certificate Authority (CA).
   tls:
     key:  /certs/kes-server.key
     cert: /certs/kes-server.cert
   
     # Specify the path to CAs used by KES for validating client certificates
     # This can alternatively be a single CA
     # KES uses these CAs in addition to the system trust store for validating client certificates.
   
     ca: /certs/CAs/
   
   # Sets access policies for KES
   # The `minio` policy grants access to the listed APIs.
   policy:
     minio:
       allow:
       - /v1/key/create/*   # You can replace these wildcard '*' with a string prefix to restrict key names
       - /v1/key/generate/* # e.g. '/minio-'
       - /v1/key/decrypt/*
       - /v1/key/bulk/decrypt
       - /v1/key/list/*
       - /v1/status
       - /v1/metrics
       - /v1/log/audit
       - /v1/log/error
       identities:
       - MINIO_API_KEY_HASH # Replace with the hash output returned from kes identity new
   
   # Specify the connection information for the Vault server.
   # The endpoint should be resolvable from the host.
   # This example assumes that Vault is configured with an AppRole ID and
   # Secret for use with KES.
   keystore:
     vault:
       endpoint: https://HOSTNAME:8200
       engine: "/path/to/engine" # Replace with the path to the K/V Engine
       version: "v1|v2" # Specify v1 or v2 depending on the version of the K/V Engine
       approle:
         id: "VAULTAPPID"     # Hashicorp Vault AppRole ID
         secret: "VAULTAPPSECRET" # Hashicorp Vault AppRole Secret ID
         retry: 15s
       status:
         ping: 10s
       # Required if Vault uses certificates signed by an unknown CA,
       # e.g. self-signed or internal (non-globally trusted).
       # Replace this value with the full path to the Vault CA certificate.
       tls:
         ca: vault-tls-CA.cert
   

   • Set MINIO_IDENTITY_HASH to the identity hash of the MinIO mTLS certificate.

     The following command computes the necessary hash:

     podman run --rm                                             \
        -v ~/minio-kes-vault/certs/certs:/certs                                \
        kes:2024-01-11T13-09-29Z tool identity of /certs/minio-kes.cert
     

   • Replace the vault.endpoint with the hostname of the Vault server(s).

   • Replace the VAULTAPPID and VAULTAPPSECRET with the appropriate Vault AppRole credentials.

2. Create the MinIO Environment File

   Create the environment file using your preferred text editor. The following example uses nano:

   nano ~/minio-kes-vault/config/minio
   

   This command assumes the minio-kes.cert, minio-kes.key, and kes-server.cert certificates are accessible at the specified location:

   MINIO_ROOT_USER=myminioadmin
   MINIO_ROOT_PASSWORD=minio-secret-key-change-me
   MINIO_VOLUMES="/mnt/data"
   
   # KES Configurations
   
   MINIO_KMS_KES_ENDPOINT=https://127.0.0.1:7373
   MINIO_KMS_KES_CERT_FILE=/certs/minio-kes.cert
   MINIO_KMS_KES_KEY_FILE=/certs/minio-kes.key
   MINIO_KMS_KES_CAPATH=/certs/server.cert
   MINIO_KMS_KES_KEY_NAME=minio-backend-default-key
   MINIO_KMS_KES_ENCLAVE=<name>
   

   MinIO uses the MINIO_KMS_KES_KEY_NAME key for the following cryptographic operations:

   • Encrypting the MinIO backend (IAM, configuration, etc.)

   • Encrypting objects using SSE-KMS if the request does not include a specific EK.

   • Encrypting objects using SSE-S3.

   MinIO uses the MINIO_KMS_KES_ENCLAVE key to define the name of the KES enclave to use.

   • Replace <name> with the name of the enclave to use.

   • If not defined, MinIO does not send any enclave information. This may result in using the default enclave for stateful KES servers.

     A KES enclave isolates its associated keys from other enclaves on a stateful KES server.

   The minio-kes certificates enable for mTLS between the MinIO deployment and the KES server only. They do not otherwise enable TLS for other client connections to MinIO.

   KES automatically creates this key if it does not already exist on the root KMS.

3) Create Pod and Containers

The commands in this section create the following resources:

• A Podman Pod to facilitate container communications

• A Container for the KES Server configured to use Hashicorp Vault as the Root KMS.

• A Container for a MinIO Server running in Single-Node Single-Drive Mode.

sudo podman pod create  \
  -p 9000:9000 -p 9001:9001 -p 7373:7373  \
  -v ~/minio-kes-vault/certs:/certs  \
  -v ~/minio-kes-vault/minio:/mnt/minio  \
  -v ~/minio-kes-vault/config:/etc/default/  \
  -n minio-kes-vault

sudo podman run -dt  \
  --cap-add IPC_LOCK  \
  --name kes-server  \
  --pod "minio-kes-vault"  \
  -e KES_SERVER=https://127.0.0.1:7373  \
  -e KES_CLIENT_KEY=/certs/kes-server.key  \
  -e KES_CLIENT_CERT=/certs/kes-server.cert  \
  quay.io/minio/kes:2024-01-11T13-09-29Z server  \
    --auth  \
    --config=/etc/default/kes-config.yaml  \

sudo podman run -dt  \
  --name minio-server  \
  --pod "minio-kes-vault"  \
  -e "MINIO_CONFIG_ENV_FILE=/etc/default/minio"  \
  quay.io/minio/minio:RELEASE.2024-02-17T01-15-57Z server  \
    --console-address ":9001"

You can verify the status of the containers using the following commands:

# Should show three pods - one for the Pod, one for KES, and one for MinIO
sudo podman container ls

If all pods are operational, you can connect to the MinIO deployment by opening your browser to http://127.0.0.1:9000 and logging in with the root credentials specified in the MinIO environment file.

4) Generate a New Encryption Key

MinIO requires that the EK exist on the root KMS before performing SSE operations using that key. Use kes key create or mc admin kms key create to create a new EK for use with SSE.

The following command uses the kes key create command to add a new External Key (EK) stored on the root KMS server for use with encrypting the MinIO backend.

sudo podman run --rm  \
  -v ~/minio-kes-vault/certs:/certs  \
  -e KES_SERVER=https://127.0.0.1:7373  \
  -e KES_CLIENT_KEY=/certs/minio-kes.key  \
  -e KES_CLIENT_CERT=/certs/minio-kes.cert  \
  kes:2024-01-11T13-09-29Z key create -k my-new-encryption-key

You can specify any key name as appropriate for your use case, such as a bucket-specific key minio-mydata-key.

5) Enable SSE-KMS for a Bucket

You can use either the MinIO Console or the MinIO mc CLI to enable bucket-default SSE-KMS with the generated key:

mc alias set local http://127.0.0.1:9000 ROOTUSER ROOTPASSWORD

mc mb local/encryptedbucket
mc encrypt set SSE-KMS encrypted-bucket-key ALIAS/encryptedbucket