Deploy Operator With Helm

Overview

Helm is a tool for automating the deployment of applications to Kubernetes clusters. A Helm chart is a set of YAML files, templates, and other files that define the deployment details. The following procedure uses a Helm Chart to install the MinIO Kubernetes Operator to a Kubernetes cluster.

Prerequisites

To install the Operator with Helm you will need the following:

An existing Kubernetes cluster.
The kubectl CLI tool on your local host, the same version as the cluster.
Helm version 3.8 or greater.
yq version 4.18.1 or greater.
Access to run kubectl commands on the cluster from your local host.

For more about Operator installation requirements, including supported Kubernetes versions and TLS certificates, see the Operator deployment prerequisites.

This procedure assumes familiarity with the referenced Kubernetes concepts and utilities. While this documentation may provide guidance for configuring or deploying Kubernetes-related resources on a best-effort basis, it is not a replacement for the official Kubernetes Documentation.

Install the MinIO Operator using Helm Charts

The following procedure installs the Operator using the MinIO Operator Chart Repository. This method supports a simplified installation path compared to the local chart installation. You can modify the Operator deployment after installation.

Important

Do not use kubectl krew or similar methods to update or manage the MinIO Operator installation. If you use Helm charts to install the Operator, you must use Helm to manage that installation.

Add the MinIO Operator Repo to Helm

MinIO maintains a Helm-compatible repository at https://operator.min.io. Add this repository to Helm:

helm repo add minio-operator https://operator.min.io

You can validate the repo contents using helm search:

helm search repo minio-operator

The response should resemble the following:

NAME                            CHART VERSION   APP VERSION     DESCRIPTION
minio-operator/minio-operator   4.3.7           v4.3.7          A Helm chart for MinIO Operator
minio-operator/operator         5.0.10          v5.0.10         A Helm chart for MinIO Operator
minio-operator/tenant           5.0.10          v5.0.10         A Helm chart for MinIO Operator

The minio-operator/minio-operator is a legacy chart and should not be installed under normal circumstances.

Install the Operator

Run the helm install command to install the Operator:

helm install \
  --namespace minio-operator \
  --create-namespace \
  operator minio-operator/operator

Verify the Operator installation

Check the contents of the specified namespace (minio-operator) to ensure all pods and services have started successfully.

kubectl get all -n minio-operator

The response should resemble the following:

NAME                                  READY   STATUS    RESTARTS   AGE
pod/console-68d955874d-vxlzm          1/1     Running   0          25h
pod/minio-operator-699f797b8b-th5bk   1/1     Running   0          25h
pod/minio-operator-699f797b8b-nkrn9   1/1     Running   0          25h

NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/console    ClusterIP   10.43.195.224   <none>        9090/TCP,9443/TCP   25h
service/operator   ClusterIP   10.43.44.204    <none>        4221/TCP            25h
service/sts        ClusterIP   10.43.70.4      <none>        4223/TCP            25h

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/console          1/1     1            1           25h
deployment.apps/minio-operator   2/2     2            2           25h

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/console-68d955874d          1         1         1       25h
replicaset.apps/minio-operator-699f797b8b   2         2         2       25h

(Optional) Enable NodePort Access to the Console

You can enable Node Port access to the service/console service to allow simplified access to the MinIO Operator. You can skip this step if you intend to configure the Operator Console service to use a Kubernetes Load Balancer, ingress, or similar control plane component that enables external access.

Edit the service/console and set the spec.ports[0].nodePort and spec.type fields as follows:
```
spec:
  ports:
  - name: http
    port: 9090
    protocol: TCP
    targetPort: 9090
    nodePort: 30090
type: NodePort
```
You can attempt to connect to the MinIO Operator Console by specifying port 30090 on any of the worker nodes in the deployment.

Retrieve the Console Access Token

The MinIO Operator uses a JSON Web Token (JWT) saved as a Kubernetes Secret for controlling access to the Operator Console.

Use the following command to retrieve the JWT for login. You must have permission within the Kubernetes cluster to read secrets:

kubectl get secret/console-sa-secret -n minio-operator -o json | jq -r ".data.token" | base64 -d

The output should resemble the following:

eyJhbGciOiJSUzI1NiIsImtpZCI6IlRtV2x3Z1RILVREaThhQm9iemFfLW95NHFHT0ZZOHFBRjlZalBRcWZiSDgifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtaW5pby1vcGVyYXRvciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjb25zb2xlLXNhLXNlY3JldCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJjb25zb2xlLXNhIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiY2M1ZjEwYzktYzU1ZC00MjNiLTgxM2MtNmU5ZDY2ZGI5NDYyIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Om1pbmlvLW9wZXJhdG9yOmNvbnNvbGUtc2EifQ.F-Pt5nU9xaugjRksWAOTShBW_eNTf8UwXvLfGxEK6l3_41NYsLgvTg5m0hYLUiYr6v2HwkEu0XzqTJbPoeSrFds8BOjeiCoP2Lmw4tRPo9tSXhAq-_elWt83YpJl-zjUpna5nVSWJWXKgj1Iga-9gw-Q63UygEcyTJ9_AwCNU9T0HdPzqccS9XrEUdsXFQxR9RwZY4TGC8K7cD9sc_OmfEiuyilRgyC_gFRvtCQfFv1DP0GKyjMGo2ffu-2Tq2U7zK5epWdqmNSvbIa0ZRoPlPedZ6nYY935lNgTIIW1oykRYrgwZZiv4CzfTH2gPswjtPc5ICtDDRUjYEhdTq3gtw

If the output includes a trailing % make sure to omit it from the result.

Log into the MinIO Operator Console

If you configured the svc/console service for access through ingress, a cluster load balancer, you can access the Console using the configured hostname and port.

If you configured the service for access through NodePorts, specify the hostname of any worker node in the cluster with that port as HOSTNAME:NODEPORT to access the Console.

Alternatively, you can use kubectl port forward to temporary forward ports for the Console:
```
kubectl port-forward svc/console -n minio-operator 9090:9090
```
You can then use http://localhost:9090 to access the MinIO Operator Console.

Once you access the Console, use the Console JWT to log in.

You can now deploy and manage MinIO Tenants using the Operator Console.

You can also deploy a tenant using Helm Charts.

Install the MinIO Operator using Local Helm Charts

The following procedure installs the Operator using a local copy of the Helm Charts. This method may support easier pre-configuration of the Operator compared to the repo-based installation

Download the Helm charts

On your local host, download the Operator Helm charts to a convenient directory:
```
curl -O https://raw.githubusercontent.com/minio/operator/master/helm-releases/operator-5.0.14.tgz
```
The chart contains a values.yaml file you can customize to suit your needs. For details on the options available in the MinIO Operator values.yaml, see Operator Helm Charts.

For more about customizations, see Helm Charts.

Deploy Operator

The following Helm command deploys the MinIO Operator using the downloaded chart:

helm install \
--namespace minio-operator \
--create-namespace \
operator operator-5.0.14.tgz

Configure Operator

Create the YAML configuration files

Use kubectl and yq to create the following files:

service.yaml:

kubectl get service console -n minio-operator -o yaml > service.yaml
yq e -i '.spec.type="NodePort"' service.yaml
yq e -i '.spec.ports[0].nodePort = PORT_NUMBER' service.yaml

Replace PORT_NUMBER with the port on which to serve the Operator GUI.

The file contents resemble the following:

operator.yaml:

kubectl get deployment minio-operator -n minio-operator -o yaml > operator.yaml
yq -i -e '.spec.replicas |= 1' operator.yaml

The file contents resemble the following:

Example operator.yaml file

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
    meta.helm.sh/release-name: minio-operator
    meta.helm.sh/release-namespace: minio-operator
  creationTimestamp: "2023-05-11T14:57:43Z"
  generation: 1
  labels:
    app.kubernetes.io/instance: minio-operator
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: operator
    app.kubernetes.io/version: v5.0.4
    helm.sh/chart: operator-5.0.4
  name: minio-operator
  namespace: minio-operator
  resourceVersion: "947"
  uid: f395171e-d17c-4645-9854-3dd92f23be59
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/instance: minio-operator
      app.kubernetes.io/name: operator
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app.kubernetes.io/instance: minio-operator
        app.kubernetes.io/name: operator
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: name
                    operator: In
                    values:
                      - minio-operator
              topologyKey: kubernetes.io/hostname
      containers:
        - args:
            - controller
          image: quay.io/minio/operator:v5.0.4
          imagePullPolicy: IfNotPresent
          name: operator
          resources:
            requests:
              cpu: 200m
              ephemeral-storage: 500Mi
              memory: 256Mi
          securityContext:
            runAsGroup: 1000
            runAsNonRoot: true
            runAsUser: 1000
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1000
        runAsGroup: 1000
        runAsNonRoot: true
        runAsUser: 1000
      serviceAccount: minio-operator
      serviceAccountName: minio-operator
      terminationGracePeriodSeconds: 30
status:
  conditions:
    - lastTransitionTime: "2023-05-11T14:57:43Z"
      lastUpdateTime: "2023-05-11T14:57:43Z"
      message: Deployment does not have minimum availability.
      reason: MinimumReplicasUnavailable
      status: "False"
      type: Available
    - lastTransitionTime: "2023-05-11T14:57:43Z"
      lastUpdateTime: "2023-05-11T14:57:44Z"
      message: ReplicaSet "minio-operator-674cf5cf78" is progressing.
      reason: ReplicaSetUpdated
      status: "True"
      type: Progressing
  observedGeneration: 1
  replicas: 2
  unavailableReplicas: 2
  updatedReplicas: 2

console-secret.yaml:

Create a console-secret.yaml file with the following contents:

apiVersion: v1
kind: Secret
metadata:
  name: console-sa-secret
  namespace: minio-operator
  annotations:
    kubernetes.io/service-account.name: console-sa
type: kubernetes.io/service-account-token

Apply the configuration to your deployment with kubectl apply:

kubectl apply -f service.yaml
kubectl apply -f operator.yaml
kubectl apply -f console-secret.yaml

To verify the installation, run the following command:

kubectl get all --namespace minio-operator

If you initialized the Operator with a custom namespace, replace minio-operator with that namespace.

The output resembles the following:

NAME                                  READY   STATUS    RESTARTS   AGE
pod/console-59b769c486-cv7zv          1/1     Running   0          81m
pod/minio-operator-7976b4df5b-rsskl   1/1     Running   0          81m

NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/console    ClusterIP   10.105.218.94    <none>        9090/TCP,9443/TCP   81m
service/operator   ClusterIP   10.110.113.146   <none>        4222/TCP,4233/TCP   81m

NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/console          1/1     1            1           81m
deployment.apps/minio-operator   1/1     1            1           81m

NAME                                        DESIRED   CURRENT   READY   AGE
replicaset.apps/console-59b769c486          1         1         1       81m
replicaset.apps/minio-operator-7976b4df5b   1         1         1       81m

Connect to the Operator Console

To connect to the Console, first retrieve the JSON Web Token (JWT) for your deployment and then forward the Console port.
1. Retrieve the JWT
  
  The Operator Console uses a JWT to authenticate and log in. The following commands retrieve the token for your deployment:
```
SA_TOKEN=$(kubectl -n minio-operator  get secret console-sa-secret -o jsonpath="{.data.token}" | base64 --decode)
echo $SA_TOKEN
```
2. Forward the Operator Console port to allow access from another host.
  
  The following command temporarily forwards the Console to port 9090:
```
kubectl --namespace minio-operator port-forward svc/console 9001:9001
```
  This command forwards the pod port 9090 to the matching port on the local machine while active in the shell. The kubectl port-forward command only functions while active in the shell session. Terminating the session closes the ports on the local machine.
3. Access the Console by navigating to http://localhost:9001 in a browser and login with the JWT.
Note

Some Kubernetes deployments may experience issues with timeouts during port-forwarding operations with the Operator Console. Select the NodePorts section to view instructions for alternative access. You can alternatively configure your preferred Ingress to grant access to the Operator Console service. See https://github.com/kubernetes/kubectl/issues/1368 for more information.

You can now deploy and manage MinIO Tenants using the Operator Console.

You can also deploy a tenant using Helm Charts.