Documentation

Deploy Operator With Helm

Overview

Helm is a tool for automating the deployment of applications to Kubernetes clusters. A Helm chart is a set of YAML files, templates, and other files that define the deployment details. The following procedure uses a Helm Chart to install the MinIO Kubernetes Operator to a Kubernetes cluster.

Prerequisites

To install the Operator with Helm you will need the following:

  • An existing Kubernetes cluster.

  • The kubectl CLI tool on your local host, the same version as the cluster.

  • Helm version 3.8 or greater.

  • yq version 4.18.1 or greater.

  • Access to run kubectl commands on the cluster from your local host.

For more about Operator installation requirements, including supported Kubernetes versions and TLS certificates, see the Operator deployment prerequisites.

This procedure assumes familiarity with the referenced Kubernetes concepts and utilities. While this documentation may provide guidance for configuring or deploying Kubernetes-related resources on a best-effort basis, it is not a replacement for the official Kubernetes Documentation.

Install the MinIO Operator using Helm Charts

The following procedure installs the Operator using the MinIO Operator Chart Repository. This method supports a simplified installation path compared to the local chart installation. You can modify the Operator deployment after installation.

Important

Do not use kubectl krew or similar methods to update or manage the MinIO Operator installation. If you use Helm charts to install the Operator, you must use Helm to manage that installation.

  1. Add the MinIO Operator Repo to Helm

    MinIO maintains a Helm-compatible repository at https://operator.min.io. Add this repository to Helm:

    helm repo add minio-operator https://operator.min.io
    

    You can validate the repo contents using helm search:

    helm search repo minio-operator
    

    The response should resemble the following:

    NAME                            CHART VERSION   APP VERSION     DESCRIPTION
    minio-operator/minio-operator   4.3.7           v4.3.7          A Helm chart for MinIO Operator
    minio-operator/operator         5.0.10          v5.0.10         A Helm chart for MinIO Operator
    minio-operator/tenant           5.0.10          v5.0.10         A Helm chart for MinIO Operator
    

    The minio-operator/minio-operator is a legacy chart and should not be installed under normal circumstances.

  2. Install the Operator

    Run the helm install command to install the Operator:

    helm install \
      --namespace minio-operator \
      --create-namespace \
      operator minio-operator/operator
    
  3. Verify the Operator installation

    Check the contents of the specified namespace (minio-operator) to ensure all pods and services have started successfully.

    kubectl get all -n minio-operator
    

    The response should resemble the following:

    NAME                                  READY   STATUS    RESTARTS   AGE
    pod/console-68d955874d-vxlzm          1/1     Running   0          25h
    pod/minio-operator-699f797b8b-th5bk   1/1     Running   0          25h
    pod/minio-operator-699f797b8b-nkrn9   1/1     Running   0          25h
    
    NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
    service/console    ClusterIP   10.43.195.224   <none>        9090/TCP,9443/TCP   25h
    service/operator   ClusterIP   10.43.44.204    <none>        4221/TCP            25h
    service/sts        ClusterIP   10.43.70.4      <none>        4223/TCP            25h
    
    NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/console          1/1     1            1           25h
    deployment.apps/minio-operator   2/2     2            2           25h
    
    NAME                                        DESIRED   CURRENT   READY   AGE
    replicaset.apps/console-68d955874d          1         1         1       25h
    replicaset.apps/minio-operator-699f797b8b   2         2         2       25h
    
  4. (Optional) Enable NodePort Access to the Console

    You can enable Node Port access to the service/console service to allow simplified access to the MinIO Operator. You can skip this step if you intend to configure the Operator Console service to use a Kubernetes Load Balancer, ingress, or similar control plane component that enables external access.

    Edit the service/console and set the spec.ports[0].nodePort and spec.type fields as follows:

    spec:
      ports:
      - name: http
        port: 9090
        protocol: TCP
        targetPort: 9090
        nodePort: 30090
    type: NodePort
    

    You can attempt to connect to the MinIO Operator Console by specifying port 30090 on any of the worker nodes in the deployment.

  5. Retrieve the Console Access Token

    The MinIO Operator uses a JSON Web Token (JWT) saved as a Kubernetes Secret for controlling access to the Operator Console.

    Use the following command to retrieve the JWT for login. You must have permission within the Kubernetes cluster to read secrets:

    kubectl get secret/console-sa-secret -n minio-operator -o json | jq -r ".data.token" | base64 -d
    

    The output should resemble the following:

    eyJhbGciOiJSUzI1NiIsImtpZCI6IlRtV2x3Z1RILVREaThhQm9iemFfLW95NHFHT0ZZOHFBRjlZalBRcWZiSDgifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJtaW5pby1vcGVyYXRvciIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJjb25zb2xlLXNhLXNlY3JldCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJjb25zb2xlLXNhIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiY2M1ZjEwYzktYzU1ZC00MjNiLTgxM2MtNmU5ZDY2ZGI5NDYyIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Om1pbmlvLW9wZXJhdG9yOmNvbnNvbGUtc2EifQ.F-Pt5nU9xaugjRksWAOTShBW_eNTf8UwXvLfGxEK6l3_41NYsLgvTg5m0hYLUiYr6v2HwkEu0XzqTJbPoeSrFds8BOjeiCoP2Lmw4tRPo9tSXhAq-_elWt83YpJl-zjUpna5nVSWJWXKgj1Iga-9gw-Q63UygEcyTJ9_AwCNU9T0HdPzqccS9XrEUdsXFQxR9RwZY4TGC8K7cD9sc_OmfEiuyilRgyC_gFRvtCQfFv1DP0GKyjMGo2ffu-2Tq2U7zK5epWdqmNSvbIa0ZRoPlPedZ6nYY935lNgTIIW1oykRYrgwZZiv4CzfTH2gPswjtPc5ICtDDRUjYEhdTq3gtw
    

    If the output includes a trailing % make sure to omit it from the result.

  6. Log into the MinIO Operator Console

    If you configured the svc/console service for access through ingress, a cluster load balancer, you can access the Console using the configured hostname and port.

    If you configured the service for access through NodePorts, specify the hostname of any worker node in the cluster with that port as HOSTNAME:NODEPORT to access the Console.

    Alternatively, you can use kubectl port forward to temporary forward ports for the Console:

    kubectl port-forward svc/console -n minio-operator 9090:9090
    

    You can then use http://localhost:9090 to access the MinIO Operator Console.

    Once you access the Console, use the Console JWT to log in.

You can now deploy and manage MinIO Tenants using the Operator Console.

You can also deploy a tenant using Helm Charts.

Install the MinIO Operator using Local Helm Charts

The following procedure installs the Operator using a local copy of the Helm Charts. This method may support easier pre-configuration of the Operator compared to the repo-based installation

  1. Download the Helm charts

    On your local host, download the Operator Helm charts to a convenient directory:

    curl -O https://raw.githubusercontent.com/minio/operator/master/helm-releases/operator-5.0.12.tgz
    

    The chart contains a values.yaml file you can customize to suit your needs. For more about customizations, see Helm Charts.

  2. Deploy Operator

    The following Helm command deploys the MinIO Operator using the downloaded chart:

    helm install \
    --namespace minio-operator \
    --create-namespace \
    operator operator-5.0.12.tgz
    
  3. Configure Operator

    1. Create the YAML configuration files

      Use kubectl and yq to create the following files:

      • service.yaml:

        kubectl get service console -n minio-operator -o yaml > service.yaml
        yq e -i '.spec.type="NodePort"' service.yaml
        yq e -i '.spec.ports[0].nodePort = PORT_NUMBER' service.yaml
        

        Replace PORT_NUMBER with the port on which to serve the Operator GUI.

        The file contents resemble the following:

        Example service.yaml file
        apiVersion: v1
        kind: Service
        metadata:
          annotations:
            meta.helm.sh/release-name: minio-operator
            meta.helm.sh/release-namespace: minio-operator
          creationTimestamp: "2023-05-11T14:57:42Z"
          labels:
            app.kubernetes.io/instance: minio-operator
            app.kubernetes.io/managed-by: Helm
            app.kubernetes.io/name: operator
            app.kubernetes.io/version: v5.0.4
            helm.sh/chart: operator-5.0.4
          name: console
          namespace: minio-operator
          resourceVersion: "907"
          uid: 9297fd97-806a-4715-8bd5-a1f6103149a8
        spec:
          clusterIP: 10.96.157.135
          clusterIPs:
            - 10.96.157.135
          internalTrafficPolicy: Cluster
          ipFamilies:
            - IPv4
          ipFamilyPolicy: SingleStack
          ports:
            - name: http
              port: 9090
              protocol: TCP
              targetPort: 9090
              nodePort: 30080
            - name: https
              port: 9443
              protocol: TCP
              targetPort: 9443
          selector:
            app.kubernetes.io/instance: minio-operator-console
            app.kubernetes.io/name: operator
          sessionAffinity: None
          type: NodePort
        status:
          loadBalancer: {}
        
      • operator.yaml:

        kubectl get deployment minio-operator -n minio-operator -o yaml > operator.yaml
        yq -i -e '.spec.replicas |= 1' operator.yaml
        

        The file contents resemble the following:

        Example operator.yaml file
        apiVersion: apps/v1
        kind: Deployment
        metadata:
          annotations:
            deployment.kubernetes.io/revision: "1"
            meta.helm.sh/release-name: minio-operator
            meta.helm.sh/release-namespace: minio-operator
          creationTimestamp: "2023-05-11T14:57:43Z"
          generation: 1
          labels:
            app.kubernetes.io/instance: minio-operator
            app.kubernetes.io/managed-by: Helm
            app.kubernetes.io/name: operator
            app.kubernetes.io/version: v5.0.4
            helm.sh/chart: operator-5.0.4
          name: minio-operator
          namespace: minio-operator
          resourceVersion: "947"
          uid: f395171e-d17c-4645-9854-3dd92f23be59
        spec:
          progressDeadlineSeconds: 600
          replicas: 1
          revisionHistoryLimit: 10
          selector:
            matchLabels:
              app.kubernetes.io/instance: minio-operator
              app.kubernetes.io/name: operator
          strategy:
            rollingUpdate:
              maxSurge: 25%
              maxUnavailable: 25%
            type: RollingUpdate
          template:
            metadata:
              creationTimestamp: null
              labels:
                app.kubernetes.io/instance: minio-operator
                app.kubernetes.io/name: operator
            spec:
              affinity:
                podAntiAffinity:
                  requiredDuringSchedulingIgnoredDuringExecution:
                    - labelSelector:
                        matchExpressions:
                          - key: name
                            operator: In
                            values:
                              - minio-operator
                      topologyKey: kubernetes.io/hostname
              containers:
                - args:
                    - controller
                  image: quay.io/minio/operator:v5.0.4
                  imagePullPolicy: IfNotPresent
                  name: operator
                  resources:
                    requests:
                      cpu: 200m
                      ephemeral-storage: 500Mi
                      memory: 256Mi
                  securityContext:
                    runAsGroup: 1000
                    runAsNonRoot: true
                    runAsUser: 1000
                  terminationMessagePath: /dev/termination-log
                  terminationMessagePolicy: File
              dnsPolicy: ClusterFirst
              restartPolicy: Always
              schedulerName: default-scheduler
              securityContext:
                fsGroup: 1000
                runAsGroup: 1000
                runAsNonRoot: true
                runAsUser: 1000
              serviceAccount: minio-operator
              serviceAccountName: minio-operator
              terminationGracePeriodSeconds: 30
        status:
          conditions:
            - lastTransitionTime: "2023-05-11T14:57:43Z"
              lastUpdateTime: "2023-05-11T14:57:43Z"
              message: Deployment does not have minimum availability.
              reason: MinimumReplicasUnavailable
              status: "False"
              type: Available
            - lastTransitionTime: "2023-05-11T14:57:43Z"
              lastUpdateTime: "2023-05-11T14:57:44Z"
              message: ReplicaSet "minio-operator-674cf5cf78" is progressing.
              reason: ReplicaSetUpdated
              status: "True"
              type: Progressing
          observedGeneration: 1
          replicas: 2
          unavailableReplicas: 2
          updatedReplicas: 2
        
      • console-secret.yaml:

        Create a console-secret.yaml file with the following contents:

        apiVersion: v1
        kind: Secret
        metadata:
          name: console-sa-secret
          namespace: minio-operator
          annotations:
            kubernetes.io/service-account.name: console-sa
        type: kubernetes.io/service-account-token
        
    2. Apply the configuration to your deployment with kubectl apply:

      kubectl apply -f service.yaml
      kubectl apply -f operator.yaml
      kubectl apply -f console-secret.yaml
      
  4. To verify the installation, run the following command:

    kubectl get all --namespace minio-operator
    

    If you initialized the Operator with a custom namespace, replace minio-operator with that namespace.

    The output resembles the following:

    NAME                                  READY   STATUS    RESTARTS   AGE
    pod/console-59b769c486-cv7zv          1/1     Running   0          81m
    pod/minio-operator-7976b4df5b-rsskl   1/1     Running   0          81m
    
    NAME               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
    service/console    ClusterIP   10.105.218.94    <none>        9090/TCP,9443/TCP   81m
    service/operator   ClusterIP   10.110.113.146   <none>        4222/TCP,4233/TCP   81m
    
    NAME                             READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/console          1/1     1            1           81m
    deployment.apps/minio-operator   1/1     1            1           81m
    
    NAME                                        DESIRED   CURRENT   READY   AGE
    replicaset.apps/console-59b769c486          1         1         1       81m
    replicaset.apps/minio-operator-7976b4df5b   1         1         1       81m
    
  5. Connect to the Operator Console

    To connect to the Console, first retrieve the JSON Web Token (JWT) for your deployment and then forward the Console port.

    1. Retrieve the JWT

      The Operator Console uses a JWT to authenticate and log in. The following commands retrieve the token for your deployment:

      SA_TOKEN=$(kubectl -n minio-operator  get secret console-sa-secret -o jsonpath="{.data.token}" | base64 --decode)
      echo $SA_TOKEN
      
    2. Forward the Operator Console port to allow access from another host.

      The following command temporarily forwards the Console to port 9090:

      kubectl --namespace minio-operator port-forward svc/console 9001:9001
      

      This command forwards the pod port 9090 to the matching port on the local machine while active in the shell. The kubectl port-forward command only functions while active in the shell session. Terminating the session closes the ports on the local machine.

    3. Access the Console by navigating to http://localhost:9001 in a browser and login with the JWT.

    Note

    Some Kubernetes deployments may experience issues with timeouts during port-forwarding operations with the Operator Console. Select the NodePorts section to view instructions for alternative access. You can alternatively configure your preferred Ingress to grant access to the Operator Console service. See https://github.com/kubernetes/kubectl/issues/1368 for more information.

NodePorts

Use the following command to identify the NodePorts configured for the Operator Console. If your local host does not have the jq utility installed, you can run the first command and locate the spec.ports section of the output.

kubectl get svc/console -n minio-operator -o json | jq -r '.spec.ports'

The output resembles the following:

[
   {
      "name": "http",
      "nodePort": 31055,
      "port": 9090,
      "protocol": "TCP",
      "targetPort": 9090
   },
   {
      "name": "https",
      "nodePort": 31388,
      "port": 9443,
      "protocol": "TCP",
      "targetPort": 9443
   }
]

Use the http or https port depending on whether you deployed the Operator with Console TLS enabled via kubectl minio init --console-tls.

Append the nodePort value to the externally-accessible IP address of a worker node in your Kubernetes cluster.

You can now deploy and manage MinIO Tenants using the Operator Console.

You can also deploy a tenant using Helm Charts.