MinIO | MinIO for SUSE Rancher

Abstract watercolor painting with dynamic red and pink brush strokes

Customers run MinIO on SUSE Rancher for three reasons.

Deploy, manage and secure AWS-like infrastructure where Kubernetes provides compute infrastructure, MinIO provides object storage, and Rancher provides unified multi-cluster management.

Turnkey multi-cluster deployment and management of DevOps tools, providing freedom to innovate without lock-in or disruption while ensuring a consistent developer experience across locations, clouds and platforms.

Running MinIO on Rancher provides control over the software stack with flexibility to avoid cloud lock-in and provide consistent object storage across hybrid and multi-cloud.

Architecture

SUSE Rancher is the only enterprise Kubernetes platform that manages all Kubernetes distros, regardless of the underlying Linux and whether they run in public clouds, private data centers or edge computing environments. The platform was designed to decrease the operational overhead of creating, managing and securing Kubernetes clusters with Rancher Kubernetes Engine (RKE) or cloud Kubernetes services, such as GKE, AKS, and EKS, or K3s at the edge. SUSE Rancher provides simple, consistent cluster operations, including provisioning, version management, visibility and diagnostics, monitoring and alerting, and centralized audit.

MinIO natively integrates with Rancher streamlining operations for large scale multi-tenant object storage as a service, across multiple clouds and at the edge. AIStor Operator fits into the Rancher toolchain, such as the kubectl CLI and the Istio service mesh, simplifying deployment and management for infrastructure and DevOps teams.

MinIO provides a consistent, performant and scalable object store for Rancher because it is Kubernetes-native by design and S3 compatible from inception. Developers can quickly deploy Amazon S3 compatible persistent storage service for all of their cloud native applications running on Rancher. The combination of MinIO on Rancher provides a powerful platform that allows applications to scale across any multi-cloud and hybrid cloud infrastructure and still be centrally managed and secured, avoiding public cloud lock-in.

The AIStor Operator integrates natively with Rancher to provide the following features across multiple clouds:

Storage Classes and Tiering‍

Tier across NVMe, HDD and Public Cloud Storage.

The key requirement to deploy MinIO at scale on Rancher is the ability tier across storage classes (NVMe, HDD, Public Cloud). This allows enterprises to manage both cost and performance.

MinIO supports automatic transition of aged objects from the fast NVMe tier to a more cost-efficient HDD tier and even cost-optimized cold Public Cloud storage tiers.

When tiering, MinIO presents a unified namespace across the tiers. Movement across the tiers is transparent to the application and is triggered by customer policies.

MinIO and Rancher enable hybrid and multi-cloud storage safely and securely by encrypting objects at the source - ensuring customers retain total control over the data. Rancher efficiently manages data across persistent block storage and cheaper object storage tiers when deployed inside the public cloud.

External Load Balancing

Load balance incoming requests with NGINX Ingress Controller.

All of MinIO’s communication is based on HTTPs, RESTFUL APIs and will support any standard, Kubernetes compatible ingress controller. This includes hardware based and software defined solutions. The most popular choice is NGINX. Use the SUSE Partner Software Catalog to install, then expose a MinIO tenant(s) using annotations.

Encryption Key Management

Manage encryption keys with HashiCorp Vault.

We recommend using Rancher secret management or HashiCorp Vault to store keys outside of the object storage system. This is a best practice for cloud native applications.

We recommend encryption be enabled by default on all buckets in production environments. MinIO uses AES-256-GCM or CHaCH20-Poly1305 encryption to protect data integrity and confidentiality with negligible performance impact.

MinIO supports all of the three server-side encryption (SSE-KMS, SSE-S3 and SSE-C) modes. SSE-S3 and SSE-KMS integrate with the KMS on the server side, whereas SSE-C uses the client supplied keys. MinIO supports setting a bucket-level default encryption key in the KMS with support for per-object encryption keys (SSE-KMS) and a broad per-deployment encryption key (SSE-S3).

MinIO relies on an external KMS to bootstrap its internal key encryption server (KES service) to enable high-performance, per object encryption. Each tenant runs its own KES server in an isolated namespace.

Identity Management

Manage identity and policy with OpenID Connect compatible Keycloak IDP.

Rancher includes a centralized user authentication proxy that integrates with an external IDP for SSO across clusters. Manage single sign-on (SSO) for Kubernetes and MinIO through a third party OpenID Connect/LDAP compatible identity provider, for example Keycloak, Okta/Auth0, Google, Facebook, ActiveDirectory and OpenLDAP. MinIO recommends OpenID Connect compatible Keycloak IDP.

Administrators can centrally manage user/application identity using an external IDP. MinIO enhances the IDP, providing AWS IAM-style users, groups, roles, policies and token service API. Enterprises gain significant architectural flexibility with an infrastructure independent and unified identity and access management (IAM) layer.

Certificate Management

Configure and manage certificates with Rancher Certificate Manager and Let’s Encrypt.

TLS is used to encrypt all traffic, including internode traffic, between applications and MinIO. TLS certificates establish the identity of network-connected resources, such as a MinIO server domain, and secure network communications.

MinIO integrates with the Rancher certificate manager so you can use the AIStor Operator to automatically configure, provision, manage and update certificates for the MinIO tenants. The tenants are completely isolated from each other in their own Kubernetes namespace with their own certificates for improved security.

Monitoring and Alerting

Track Metrics and issue alerts using Rancher Monitoring or Grafana.

MinIO recommends using Prometheus-compatible systems for monitoring and alerting on MinIO Rancher instances. MinIO publishes every object storage related Prometheus metric imaginable, from bucket capacity to access metrics. Those metrics can be collected and visualized in any Prometheus-compatible tool or the MinIO Console.

External monitoring solutions scrape the MinIO Prometheus endpoint at regular intervals. MinIO recommends either Grafana or the platform monitoring components installed in the `rancher-monitoring` project to connect to MinIO. These same tools can also be used to establish baselines and set alert thresholds for notifications, which can then be routed via Alertmanager to a notification platform such as PagerDuty, Freshservice or even SNMP.

Logging and Auditing

Output logs to an Elastic Stack for analysis.

Enabling MinIO auditing generates a log for every operation on the object storage cluster. In addition to the audit log, MinIO also logs console errors for operational troubleshooting purposes.

MinIO supports outputting logs to the Elastic Stack (or third parties) for analysis and alerting. Rancher includes the Banzai Cloud Logging Operator to collect container and application logs. To streamline operations, we recommend using the same logging and audit tool for Rancher and MinIO.