Documentation

Documentation

Expand a MinIO Tenant

Table of Contents

This procedure documents expanding the available storage capacity of an existing MinIO tenant by deploying an additional pool of MinIO pods in the Kubernetes infrastructure.

Prerequisites

MinIO Kubernetes Operator and Plugin

This procedures on this page requires a valid installation of the MinIO Kubernetes Operator and assumes the local host has a matching installation of the MinIO Kubernetes Operator. This procedure assumes the latest stable Operator and Plugin version 4.5.2.

See Deploy the MinIO Operator for complete documentation on deploying the MinIO Operator.

You can install the MinIO plugin using either the Kubernetes Krew plugin manager or manually by downloading and installing the plugin binary to your local host:

Krew is a kubectl plugin manager developed by the Kubernetes SIG CLI group. See the krew installation documentation for specific instructions. You can use the Krew plugin for Linux, MacOS, and Windows operating systems.

You can use Krew to install the MinIO kubectl plugin using the following commands:

kubectl krew update
kubectl krew install minio

You can validate the installation of the MinIO plugin using the following command:

kubectl minio version

The output should match 4.5.2.

You can download the MinIO kubectl plugin to your local system path. The kubectl CLI automatically discovers and runs compatible plugins.

The following code downloads the latest stable version 4.5.2 of the MinIO Kubernetes plugin and installs it to the system path:

curl https://github.com/minio/operator/releases/download/4.5.2/kubectl-minio_4.5.2_linux_amd64 -o kubectl-minio
chmod +x kubectl-minio
mv kubectl-minio /usr/local/bin/

The mv command above may require sudo escalation depending on the permissions of the authenticated user.

Run the following command to verify installation of the plugin:

kubectl minio version

The output should display the Operator version as 4.5.2.

You can download the MinIO kubectl plugin to your local system path. The kubectl CLI automatically discovers and runs compatible plugins.

The following PowerShell command downloads the latest stable version 4.5.2 of the MinIO Kubernetes plugin and installs it to the system path:

Invoke-WebRequest -Uri "https://github.com/minio/operator/releases/download/v4.5.2/kubectl-minio_4.5.2_windows_amd64.exe" -OutFile "C:\kubectl-plugins\kubectl-minio.exe"

Ensure the path to the plugin folder is included in the Windows PATH.

Run the following command to verify installation of the plugin:

kubectl minio version

The output should display the Operator version as 4.5.2.

Available Worker Nodes

MinIO deploys additional minio server pods as part of the new Tenant pool. The Kubernetes cluster must have sufficient available worker nodes on which to schedule the new pods.

The MinIO Operator provides configurations for controlling pod affinity and anti-affinity to direct scheduling to specific workers.

Locally Attached Drives

MinIO strongly recommends using locally attached drives on each node intended to support the new tenant pool to ensure optimal performance. MinIO’s strict read-after-write and list-after-write consistency model requires local disk filesystems (xfs, ext4, etc.).

MinIO automatically generates Persistent Volume Claims (PVC) as part of deploying a MinIO Tenant. The Operator generates one PVC for each volume in the new pool.

This procedure uses the MinIO DirectPV driver to automatically provision Persistent Volumes from locally attached drives to support the generated PVC. See the DirectPV Documentation for installation and configuration instructions.

For clusters which cannot deploy MinIO DirectPV, Local Persistent Volumes.

The following tabs provide example YAML objects for a local persistent volume and a supporting StorageClass:

The following YAML describes a Local Persistent Volume:

apiVersion: v1
kind: PersistentVolume
metadata:
   name: <PV-NAME>
spec:
   capacity:
      storage: 1Ti
   volumeMode: Filesystem
   accessModes:
   - ReadWriteOnce
   persistentVolumeReclaimPolicy: Retain
   storage-class: <STORAGE-CLASS>
   local:
      path: <PATH-TO-DISK>
   nodeAffinity:
      required:
         nodeSelectorTerms:
         - matchExpressions:
            - key: kubernetes.io/hostname
               operator: In
               values:
               - <NODE-NAME>

Replace values in brackets <VALUE> with the appropriate value for the local drive.

The following YAML describes a StorageClass that meets the requirements for a MinIO Tenant:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
   name: minio-local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

The storage class must have volumeBindingMode: WaitForFirstConsumer. Ensure all Persistent Volumes provisioned to support the MinIO Tenant use this storage class.

Procedure (CLI)

1) Expand the MinIO Tenant

Use the kubectl minio tenant expand command to create the MinIO Tenant.

The following example expands a MinIO Tenant with a Pool consisting of 4 Nodes with 4 locally-attached drives of 1Ti each:

kubectl minio tenant expand minio-tenant-1   \
  --servers                 4                \
  --volumes                 16               \
  --capacity                16Ti             \
  --storage-class           local-storage    \
  --namespace               minio-tenant-1

The following table explains each argument specified to the command:

Argument

Description

minio-tenant-1

The name of the MinIO Tenant which the command expands with the new pool.

--servers

The number of minio servers to deploy in the new Tenant Pool across the Kubernetes cluster.

--volumes

The number of volumes in the new Tenant Pool. kubectl minio determines the number of volumes per server by dividing volumes by servers.

--capacity

The total capacity of the Tenant Pool. kubectl minio determines the capacity of each volume by dividing capacity by volumes.

--storage-class

The Kubernetes StorageClass to use when creating each PVC.

--namespace

The Kubernetes namespace of the existing MinIO Tenant to which to add the new Tenant pool.

2) Validate the Expanded MinIO Tenant

Use the kubectl minio tenant info command to return a summary of the MinIO Tenant, including the new Pool:

kubectl minio tenant info minio-tenant-1 \
  --namespace minio-tenant-1