Managing Drives

• Working DirectPV plugin.

  To install the plugin, refer to the plugin installation guide.

• Working DirectPV CSI driver in Kubernetes.

  To install the driver, refer to the driver installation guide.

DirectPV must have access to drives to provision volumes. This involves a two step process:

1. Run the discover command.

   The discover command probes for eligible drives from DirectPV nodes and stores drive information in a YAML file. Examine the generated YAML file and set the select field to yes for each drive you want to add. If you do not want to add a particular drive, set its select value to no.

   DirectPV sets the select field to yes for all discovered drives by default. You must manually deselect any discovered drives that DirectPV should not format and use. Review the generated YAML file carefully.

   Below is an example of the discover command:

   # Probe and save drive information to drives.yaml file.
   $ kubectl directpv discover
   
    Discovered node 'master' ✔
    Discovered node 'node1' ✔
   
   ┌─────────────────────┬────────┬───────┬─────────┬────────────┬──────┬───────────┬─────────────┐
   │ ID                  │ NODE   │ DRIVE │ SIZE    │ FILESYSTEM │ MAKE │ AVAILABLE │ DESCRIPTION │
   ├─────────────────────┼────────┼───────┼─────────┼────────────┼──────┼───────────┼─────────────┤
   │ 252:16$ud8mwCjPT... │ master │ vdb   │ 512 MiB │ -          │ -    │ YES       │ -           │
   │ 252:16$gGz4UIuBj... │ node1  │ vdb   │ 512 MiB │ -          │ -    │ YES       │ -           │
   └─────────────────────┴────────┴───────┴─────────┴────────────┴──────┴───────────┴─────────────┘
   
   Generated 'drives.yaml' successfully.
   
   # Show generated drives.yaml file.
   $ cat drives.yaml
   version: v1
   nodes:
       - name: master
         drives:
           - id: 252:16$ud8mwCjPTH8147TysmiQ2GGLpffqUht6bz7NtHqReJo=
             name: vdb
             size: 536870912
             make: ""
             select: "yes"
       - name: node1
         drives:
           - id: 252:16$gGz4UIuBjQlO1KibOv7bZ+kEDk3UCeBneN/UJdqdQl4=
             name: vdb
             size: 536870912
             make: ""
             select: "yes"
   

2. Run the init command.

   The init command creates a request to add the selected drives in the YAML file generated using the discover command in the previous step.

   This process wipes out all data on the selected drives. Incorrect drive selection will result in permanent data loss.

   Before running this command, modify the YAML file to mark any undesired drives as no for the select field.

   Below is an example of init command:

   $ kubectl directpv init drives.yaml
   
   
    ███████████████████████████████████████████████████████████████████████████ 100%
   
    Processed initialization request 'c24e22f5-d582-49ba-a883-2ce56909904e' for node 'master' ✔
    Processed initialization request '7e38a453-88ed-412c-b146-03eef37b23bf' for node 'node1' ✔
   
   ┌──────────────────────────────────────┬────────┬───────┬─────────┐
   │ REQUEST_ID                           │ NODE   │ DRIVE │ MESSAGE │
   ├──────────────────────────────────────┼────────┼───────┼─────────┤
   │ c24e22f5-d582-49ba-a883-2ce56909904e │ master │ vdb   │ Success │
   │ 7e38a453-88ed-412c-b146-03eef37b23bf │ node1  │ vdb   │ Success │
   └──────────────────────────────────────┴────────┴───────┴─────────┘
   

Refer to the discover command and the init command for more details and optional parameters.

To get information about drives from DirectPV, run the list drives command.

$ kubectl directpv list drives
┌────────┬──────┬──────┬─────────┬─────────┬─────────┬────────┐
│ NODE   │ NAME │ MAKE │ SIZE    │ FREE    │ VOLUMES │ STATUS │
├────────┼──────┼──────┼─────────┼─────────┼─────────┼────────┤
│ master │ vdb  │ -    │ 512 MiB │ 506 MiB │ -       │ Ready  │
│ node1  │ vdb  │ -    │ 512 MiB │ 506 MiB │ -       │ Ready  │
└────────┴──────┴──────┴─────────┴─────────┴─────────┴────────┘

Refer to the list drives command for more information.

Drives are labeled to set custom tagging which can be used in volume provisioning.

# Set label 'tier' key to 'hot' value.
$ kubectl directpv label drives tier=hot

# Remove label 'tier'.
$ kubectl directpv label drives tier-

Once set, use labels to schedule drives.

Refer to the label drives command for more information.

Replace a faulty drive with a new drive on a same node. In this process, all volumes in the faulty drive are moved to the new drive then faulty drive is removed from DirectPV. Currently, DirectPV does not support moving data on the volume to the new drive. Use the replace.sh script to perform drive replacement.

Below is an example:

# Replace 'sdd' drive by 'sdf' drive on 'node1' node
$ replace.sh sdd sdf node1

Drives that do not contain any volumes can be removed.

# Remove drive 'vdb' from 'node1' node
$ kubectl directpv remove --drives=vdb --nodes=node1

Refer to the remove command for more information.

DirectPV supports using the xfs_repair utility to attempt to repair managed drives that report XFS filesystem errors, faults, or corruption. This command has no guarantee of success or complete recovery.

The repair command creates one-time Kubernetes Job with the pod name as repair-<DRIVE-ID>. Kubernetes automatically removes this job five minutes after completion.

Progress and status of the drive repair can be viewed using kubectl log command.

Before beginning a repair, you must first suspend the drive.

Use the repair.sh script to repair a faulty drive.

By Kubernetes design, a StatefulSet workload is active only if all of its pods are in running state. A faulty drive prevents the StatefulSet from starting up.

DirectPV provides a workaround to suspend failed drives which mounts the respective volumes on empty /var/lib/directpv/tmp directory with read-only access. This can be done by executing the suspend drives command.

kubectl directpv suspend drives af3b8b4c-73b4-4a74-84b7-1ec30492a6f0

Once fixed, return to normal drive functions by resuming the drive. Upon resuming, the corresponding volumes return to using the respective allocated drives. This can be done by using the resume drives command.

kubectl directpv resume drives af3b8b4c-73b4-4a74-84b7-1ec30492a6f0