Install DirectPV

Before beginning a production deployment, ensure the following steps are completed, if applicable:

• If using a private registry, make all the images listed in air-gapped installation images available in the private registry.

• If using seccomp on the system, load the DirectPV seccomp policy on all nodes.

  For more detailed instructions on seccomp, refer to the Kubernetes documentation

• If using AppArmor on the system, load the DirectPV apparmor profile on all nodes.

  For more detailed instructions on AppArmor in Kubernetes environment, refer to the Kubernetes documentation.

Install the DirectPV plugin in your local environment to manage the DirectPV CSI Driver in your Kubernetes cluster. You can install using krew or as a binary.

The latest DirectPV plugin is available in the Krew repository.

1. Update Krew to download the latest version of the plugin.

   kubectl krew update
   

2. Install DirectPV to your krew installation directory (default: $HOME/.krew).

   kubectl krew install directpv
   

3. Run kubectl directpv --version to verify DirectPV installed correctly.

   If you receive the error Error: unknown command "directpv" for "kubectl", you may need to add $HOME/.krew/bin to your $PATH.

The plugin binary name starts with kubectl-directpv and is available at https://github.com/minio/directpv/releases/latest. Download the binary for your operating system and architecture. You may need to move the file to a location available to your system path.

Refer to the documentation for your operating system for instructions on how to make a binary file executable and how to run the file. Detailed instructions for every available operating system are out of scope for this documentation.

Below is an example for GNU/Linux on amd64 architecture:

# Download DirectPV plugin.
$ release=$(curl -sfL "https://api.github.com/repos/minio/directpv/releases/latest" | awk '/tag_name/ { print substr($2, 3, length($2)-4) }')
$ curl -fLo kubectl-directpv https://github.com/minio/directpv/releases/download/v${release}/kubectl-directpv_${release}_linux_amd64
# Make the binary executable.
$ chmod a+x kubectl-directpv
$ mv kubectl-directpv /usr/local/bin/kubectl-directpv

Install the DirectPV Driver to your Kubernetes deployment.

A standard installation uses default options and installs DirectPV on all nodes.

• Kubernetes >= v1.18 on GNU/Linux on amd64.

• If you use a private registry, the below images must be pushed into your registry.

  You may use a helper script to do this.

  • quay.io/minio/csi-node-driver-registrar:v2.8.0
  • quay.io/minio/csi-provisioner:v3.5.0 (for Kubernetes >= v1.20)
  • quay.io/minio/csi-provisioner:v2.2.0-go1.18 (for kubernetes < v1.20)
  • quay.io/minio/livenessprobe:v2.10.0
  • quay.io/minio/csi-resizer:v1.8.0
  • quay.io/minio/directpv:latest

• If seccomp is enabled, load DirectPV seccomp profile on nodes where you want to install DirectPV and use --seccomp-profile flag to kubectl directpv install command.

  For more information, refer to the Kubernetes documentation on seccomp.

• If apparmor is enabled, load DirectPV apparmor profile on nodes where you want to install DirectPV and use --apparmor-profile flag to kubectl directpv install command.

  For more information, refer to the Kubernetes documentation on apparmor.

• Enabled ExpandCSIVolumes feature gate for volume expansion feature.

• Review the driver specification documentation.

• For Red Hat OpenShift users, refer to the OpenShift specific documentation for configuration prior to installing DirectPV.

The installation process creates a new storage class named directpv-min-io. You can provision DirectPV volumes by using this storage class as the storageClassName in PodSpec.VolumeClaimTemplates.

For an example of using directpv-min-io, see the MinIO example on GitHub.

Refer to the CLI Guide for more helpers on the following commands.

Install the directpv-min-io CSI driver on all nodes in the kubernetes cluster.

kubectl directpv install

• DirectPV components install in the namespace directpv.
• Specify an alternate kubeconfig with kubectl directpv --kubeconfig /path/to/kubeconfig.
• The DirectPV driver requires the Role Based Access Control (RBAC) listed in the specification document.
• The DirectPV driver runs in privileged mode, required to mount, unmount, and format drives.
• The daemonset used by DirectPV requires the following open ports:
  • 10443 for metrics
  • Port 30443 for readiness handlers

List all available drives in the kubernetes cluster. DirectPV generates an init config file (default: drives.yaml) you can use to initialize these drives.

kubectl directpv discover

Modify the file as needed to remove any drives that DirectPV should not control.

Initialize the drives selected in drives.yaml

kubectl directpv init drives.yaml

To verify the installation, use the following command to list information about the drives that were formatted and added to DirectPV.

kubectl directpv info

The following command downloads and runs an install.sh script file to perform a standard installation of DirectPV on all nodes.

curl -sfL https://github.com/minio/directpv/raw/master/docs/tools/install.sh | sh - apply

Push the following images to your private registry:

• quay.io/minio/csi-node-driver-registrar:v2.6.3
• quay.io/minio/csi-provisioner:v3.4.0
• quay.io/minio/livenessprobe:v2.9.0
• quay.io/minio/csi-resizer:v1.7.0
• quay.io/minio/directpv:latest

Notes:

• If you use a Kubernetes version earlier than v1.20, you need to push quay.io/minio/csi-provisioner:v2.2.0-go1.18

The following shell script performs the above steps:

/bin/bash -e

# set this to private registry URL (the URL should NOT include http or https)
if [ -z $PRIVATE_REGISTRY_URL ]; then "PRIVATE_REGISTRY_URL env var should be set"; fi

images[0]=quay.io/minio/csi-node-driver-registrar:v2.6.3
images[1]=quay.io/minio/csi-provisioner:v3.4.0
images[2]=quay.io/minio/livenessprobe:v2.9.0
images[3]=quay.io/minio/csi-resizer:v1.7.0
images[4]=quay.io/minio/directpv:$(curl -s "https://api.github.com/repos/minio/directpv/releases/latest" | grep tag_name | sed -E 's/.*"([^"]+)".*/\1/')

function privatize(){ echo $1 | sed "s#quay.io#${PRIVATE_REGISTRY_URL}#g"; }
function pull_tag_push(){ docker pull $1 &&  docker tag $1 $2 && docker push $2; }
for image in ${images[*]}; do pull_tag_push $image $(privatize $image); done

DirectPV requires different configuration options when deployed to OpenShift. Use the following command to install DirectPV on OpenShift with appropriate settings:

$ kubectl directpv install --openshift

To install DirectPV only on selected nodes, use the --node-selector flag to specify the desired nodes:

kubectl directpv info
# Install DirectPV on nodes having label 'group-name' key and 'bigdata' value
$ kubectl directpv install --node-selector group-name=bigdata

To install DirectPV on tainted nodes, use the --toleration flag.

The following example installs DirectPV on tainted nodes by tolerating ‘key1’ key where the value of the key is ‘PVs’ value with ‘NoSchedule’ effect

kubectl directpv install --tolerations key1=PVs:NoSchedule

The following example installs DirectPV on tainted nodes by tolerating the existence of the ‘key2’ key (regardless of any value assigned to the key), with the ‘NoExecute’ effect.

$ kubectl directpv install --tolerations key2:NoExecute

To install on a non-standard kubelet directory, set the KUBELET_DIR_PATH environment variable before starting the installation.

export KUBELET_DIR_PATH=/path/to/my/kubelet/dir
kubectl directpv install

To install using generated manifests file, use the following command.

curl -sfL https://github.com/minio/directpv/raw/master/docs/tools/install.sh | sh - apply