MinIO for Google
Kubernetes Engine

Storage Classes and Tiering

Tier across GKE Standard SSD, Google Cloud Storage and GCS for Data Archiving.

A key requirement for deploying MinIO at scale on GKE and GCP is the ability to transition objects across Google Cloud storage types. Specifically, you want support for cost-optimized "Hot-Warm" and "Hot-Cold" deployment topologies.

MinIO can use a GCS bucket as a remote tier for automatic transition of aged objects based on user-configured rules. For example, you can create one rule that transitions objects to a tier with the GCS Standard storage class, while another rule transitions objects to a tier with the GCS Nearline storage class.

MinIO supports transitioning an object once - so while you cannot configure "waterfall" or "chain" transition from MinIO -> Standard -> Nearline, you can configure multiple rules per bucket using prefix and object tags to apply granular transition behavior to the preferred remote tier. MinIO's only requirement is that the remote storage must support immediate retrieval of objects - no rehydration, latency, or wait times.

MinIO tiering requires no client-side logic changes. Your clients can continue to request an object through MinIO, and MinIO handles retrieving the object from GCS and returning it transparently. MinIO also supports using the S3 restore API for returning objects back to the "hot" MinIO deployment.

MinIO's tiering capability extends to hybrid cloud environments where the MinIO JBOD/JBOF deployment acts as the performance-optimized "hot" tier on the private cloud, while GCS provides cost-optimized "warm" and "cold" tiers. Leverage MinIO TLS and Server-Side Encryption to further protect all data in both clouds, at rest and in flight.

External Load Balancing

Load balance incoming requests with GCP Cloud Load Balancing.

The AIStor Operator integrates tightly with GCP Cloud Load Balancing (CLB) to provide automatic load balancing and routing service across multiple MinIO tenants for applications accessing the storage service from outside of GKE. Exposing a MinIO tenant to external traffic can be done by simply adding annotations to a MinIO tenant.

Encryption Key Management

Manage encryption keys with GCP Cloud Key Management.

MinIO recommends using Google Cloud Key Management to store keys outside of the object storage system. For those with more stringent security requirements or for consistency purposes, MinIO integrates with a number of external Key Management Services that operate outside of GCP.

For all production environments we recommend enabling encryption on all buckets by default. MinIO uses AES-256-GCM or ChaCha20-Poly1305 encryption to protect data integrity and confidentiality with negligible performance impact.

MinIO supports setting a bucket-level default encryption key in the KMS with support for per-object encryption keys (SSE-KMS) and a broad per-deployment encryption key (SSE-S3).

MinIO will use this KMS to bootstrap its internal key encryption server (KES service) to enable high-performance, per object encryption. Each tenant runs its own KES server in an isolated namespace.

Identity Services

Manage identity and policy with GCP Cloud Identity.

When running MinIO on GCP GKE, customers can manage single sign-on (SSO) through Google’s hosted GCP Cloud Identity or third party OpenID Connect/LDAP compatible identity providers like Okta/Auth0, Google, Facebook, Keycloak, ActiveDirectory and OpenLDAP.

A single, centralized IDP allows administrators to add, change privileges for, or eliminate a user, service account, or group once - and have it be enforced across all public cloud, private cloud and edge MinIO servers. The ability to have a unified identity and access management (IAM) layer independent of the infrastructure provides significant architectural flexibility.

Certificate Management

Configure and manage certificates with GKE Managed Certificates.

All traffic from the application to MinIO, including internode traffic, is encrypted with TLS. TLS certificates are used to secure network communications and establish the identity of network-connected resources, such as a MinIO Server.

MinIO integrates with GKE Managed Certificates to configure, provision, manage and update certificates for the MinIO tenants. The tenants are completely isolated from each other in their own Kubernetes namespace with their own certificates for improved security.

Monitoring and Alerting

Track metrics and issue alerts using Google Cloud Stackdriver.

GCP provides robust monitoring capabilities for GKE using Google Cloud Stackdriver. We recommend using Google Cloud Stackdriver as a Prometheus-compatible system for monitoring and alerting when deploying MinIO on GKE. The reason for this recommendation is that MinIO publishes every object storage related Prometheus metric imaginable, from bucket capacity to access metrics. Those metrics can be collected and visualized in any Prometheus-compatible tool (Stackdriver being native to GCP) or in the MinIO Console.

Monitoring services scrape the MinIO Prometheus endpoint at regular intervals. These same tools can also be used to establish baselines and set alert thresholds for notifications, which can then be routed to a notification platform such as PagerDuty or Freshservice.