Install DirectPV
Prerequisites
Production Readiness Checklist
Before beginning a production deployment, ensure the following steps are completed, if applicable:
-
If using a private registry, make all the images listed in air-gapped installation images available in the private registry.
-
If using seccomp on the system, load the DirectPV seccomp policy on all nodes.
For more detailed instructions on seccomp, refer to the Kubernetes documentation
-
If using AppArmor on the system, load the DirectPV apparmor profile on all nodes.
For more detailed instructions on AppArmor in Kubernetes environment, refer to the Kubernetes documentation.
Plugin Installation
Install the DirectPV plugin in your local environment to manage the DirectPV CSI Driver in your Kubernetes cluster.
You can install using krew
or as a binary.
krew
Install DirectPV Plugin with The latest DirectPV plugin is available in the Krew
repository.
-
Update
Krew
to download the latest version of the plugin.kubectl krew update
-
Install DirectPV to your krew installation directory (default:
$HOME/.krew
).kubectl krew install directpv
-
Run
kubectl directpv --version
to verify DirectPV installed correctly.If you receive the error
Error: unknown command "directpv" for "kubectl"
, you may need to add$HOME/.krew/bin
to your$PATH
.
Install DirectPV Plugin as a binary
The plugin binary name starts with kubectl-directpv
and is available at https://github.com/minio/directpv/releases/latest.
Download the binary for your operating system and architecture.
You may need to move the file to a location available to your system path.
Refer to the documentation for your operating system for instructions on how to make a binary file executable and how to run the file. Detailed instructions for every available operating system are out of scope for this documentation.
Below is an example for GNU/Linux
on amd64
architecture:
# Download DirectPV plugin.
$ release=$(curl -sfL "https://api.github.com/repos/minio/directpv/releases/latest" | awk '/tag_name/ { print substr($2, 3, length($2)-4) }')
$ curl -fLo kubectl-directpv https://github.com/minio/directpv/releases/download/v${release}/kubectl-directpv_${release}_linux_amd64
# Make the binary executable.
$ chmod a+x kubectl-directpv
$ mv kubectl-directpv /usr/local/bin/kubectl-directpv
kubectl-directpv
instead of kubectl directpv
.
Driver Installation
Install the DirectPV Driver to your Kubernetes deployment.
Standard Installation
A standard installation uses default options and installs DirectPV on all nodes.
Prerequisites
-
Kubernetes >= v1.18 on GNU/Linux on amd64.
-
If you use a private registry, the below images must be pushed into your registry.
You may use a helper script to do this.
- quay.io/minio/csi-node-driver-registrar:v2.8.0
- quay.io/minio/csi-provisioner:v3.5.0 (for Kubernetes >= v1.20)
- quay.io/minio/csi-provisioner:v2.2.0-go1.18 (for kubernetes < v1.20)
- quay.io/minio/livenessprobe:v2.10.0
- quay.io/minio/csi-resizer:v1.8.0
- quay.io/minio/directpv:latest
-
If
seccomp
is enabled, load DirectPV seccomp profile on nodes where you want to install DirectPV and use--seccomp-profile
flag tokubectl directpv install
command.For more information, refer to the Kubernetes documentation on seccomp.
-
If
apparmor
is enabled, load DirectPV apparmor profile on nodes where you want to install DirectPV and use--apparmor-profile
flag tokubectl directpv install
command.For more information, refer to the Kubernetes documentation on apparmor.
-
Enabled
ExpandCSIVolumes
feature gate for volume expansion feature. -
Review the driver specification documentation.
-
For Red Hat OpenShift users, refer to the OpenShift specific documentation for configuration prior to installing DirectPV.
Procedure
The installation process creates a new storage class named directpv-min-io
.
You can provision DirectPV volumes by using this storage class as the storageClassName
in PodSpec.VolumeClaimTemplates
.
For an example of using directpv-min-io
, see the MinIO example on GitHub.
Refer to the CLI Guide for more helpers on the following commands.
Install the driver
Install the directpv-min-io
CSI driver on all nodes in the kubernetes cluster.
kubectl directpv install
- DirectPV components install in the namespace
directpv
. - Specify an alternate kubeconfig with
kubectl directpv --kubeconfig /path/to/kubeconfig
. - The DirectPV driver requires the Role Based Access Control (RBAC) listed in the specification document.
- The DirectPV driver runs in
privileged
mode, required to mount, unmount, and format drives. - The daemonset used by DirectPV requires the following open ports:
10443
for metrics- Port
30443
for readiness handlers
kubelet
directory, see the custom installation section below.
List discovered drives
List all available drives in the kubernetes cluster.
DirectPV generates an init
config file (default: drives.yaml
) you can use to initialize these drives.
kubectl directpv discover
Modify the file as needed to remove any drives that DirectPV should not control.
Initialize the drives
Initialize the drives selected in drives.yaml
kubectl directpv init drives.yaml
Verify installation
To verify the installation, use the following command to list information about the drives that were formatted and added to DirectPV.
kubectl directpv info
Install with a Script
The following command downloads and runs an install.sh
script file to perform a standard installation of DirectPV on all nodes.
curl -sfL https://github.com/minio/directpv/raw/master/docs/tools/install.sh | sh - apply
Air-gapped Installation (Private Registry)
Push the following images to your private registry:
- quay.io/minio/csi-node-driver-registrar:v2.6.3
- quay.io/minio/csi-provisioner:v3.4.0
- quay.io/minio/livenessprobe:v2.9.0
- quay.io/minio/csi-resizer:v1.7.0
- quay.io/minio/directpv:latest
Notes:
- If you use a Kubernetes version earlier than v1.20, you need to push
quay.io/minio/csi-provisioner:v2.2.0-go1.18
The following shell script performs the above steps:
/bin/bash -e
# set this to private registry URL (the URL should NOT include http or https)
if [ -z $PRIVATE_REGISTRY_URL ]; then "PRIVATE_REGISTRY_URL env var should be set"; fi
images[0]=quay.io/minio/csi-node-driver-registrar:v2.6.3
images[1]=quay.io/minio/csi-provisioner:v3.4.0
images[2]=quay.io/minio/livenessprobe:v2.9.0
images[3]=quay.io/minio/csi-resizer:v1.7.0
images[4]=quay.io/minio/directpv:$(curl -s "https://api.github.com/repos/minio/directpv/releases/latest" | grep tag_name | sed -E 's/.*"([^"]+)".*/\1/')
function privatize(){ echo $1 | sed "s#quay.io#${PRIVATE_REGISTRY_URL}#g"; }
function pull_tag_push(){ docker pull $1 && docker tag $1 $2 && docker push $2; }
for image in ${images[*]}; do pull_tag_push $image $(privatize $image); done
Install on OpenShift
DirectPV requires different configuration options when deployed to OpenShift. Use the following command to install DirectPV on OpenShift with appropriate settings:
$ kubectl directpv install --openshift
Custom Installation
Install on Selected Nodes
To install DirectPV only on selected nodes, use the --node-selector
flag to specify the desired nodes:
kubectl directpv info
# Install DirectPV on nodes having label 'group-name' key and 'bigdata' value
$ kubectl directpv install --node-selector group-name=bigdata
Install on Tainted Nodes
To install DirectPV on tainted nodes, use the --toleration
flag.
The following example installs DirectPV on tainted nodes by tolerating ‘key1’ key where the value of the key is ‘PVs’ value with ‘NoSchedule’ effect
kubectl directpv install --tolerations key1=PVs:NoSchedule
The following example installs DirectPV on tainted nodes by tolerating the existence of the ‘key2’ key (regardless of any value assigned to the key), with the ‘NoExecute’ effect.
$ kubectl directpv install --tolerations key2:NoExecute
kubelet
Directory
Install on Non-standard To install on a non-standard kubelet
directory, set the KUBELET_DIR_PATH
environment variable before starting the installation.
export KUBELET_DIR_PATH=/path/to/my/kubelet/dir
kubectl directpv install
Install by Generating DirectPV Manifests
To install using generated manifests file, use the following command.
curl -sfL https://github.com/minio/directpv/raw/master/docs/tools/install.sh | sh -s - apply