Documentation

Documentation

Deploy MinIO: Single-Node Multi-Drive

The procedures on this page cover deploying MinIO in a Single-Node Multi-Drive (SNMD) configuration. SNMD deployments provide drive-level reliability and failover/recovery with performance and scaling limitations imposed by the single node.

For production environments, MinIO strongly recommends deploying with the Multi-Node Multi-Drive (Distributed) topology for enterprise-grade performance, availability, and scalability.

Prerequisites

Local JBOD Storage with Sequential Mounts

MinIO strongly recommends direct-attached JBOD arrays with XFS-formatted disks for best performance.

  • Direct-Attached Storage (DAS) has significant performance and consistency advantages over networked storage (NAS, SAN, NFS).

  • Deployments using non-XFS filesystems (ext4, btrfs, zfs) tend to have lower performance while exhibiting unexpected or undesired behavior.

  • RAID or similar technologies do not provide additional resilience or availability benefits when used with distributed MinIO deployments, and typically reduce system performance.

Ensure all server drives for which you intend MinIO to use are of the same type (NVMe, SSD, or HDD) with identical capacity (e.g. 12 TB). MinIO does not distinguish drive types and does not benefit from mixed storage types. Additionally. MinIO limits the size used per drive to the smallest drive in the deployment. For example, if the deployment has 15 10TB drives and 1 1TB drive, MinIO limits the per-drive capacity to 1TB.

MinIO requires using expansion notation {x...y} to denote a sequential series of drives when creating the new deployment, where all nodes in the deployment have an identical set of mounted drives. MinIO also requires that the ordering of physical drives remain constant across restarts, such that a given mount point always points to the same formatted drive. MinIO therefore strongly recommends using /etc/fstab or a similar file-based mount configuration to ensure that drive ordering cannot change after a reboot. For example:

$ mkfs.xfs /dev/sdb -L DISK1
$ mkfs.xfs /dev/sdc -L DISK2
$ mkfs.xfs /dev/sdd -L DISK3
$ mkfs.xfs /dev/sde -L DISK4

$ nano /etc/fstab

  # <file system>  <mount point>  <type>  <options>         <dump>  <pass>
  LABEL=DISK1      /mnt/disk1     xfs     defaults,noatime  0       2
  LABEL=DISK2      /mnt/disk2     xfs     defaults,noatime  0       2
  LABEL=DISK3      /mnt/disk3     xfs     defaults,noatime  0       2
  LABEL=DISK4      /mnt/disk4     xfs     defaults,noatime  0       2

You can then specify the entire range of drives using the expansion notation /mnt/disk{1...4}. If you want to use a specific subfolder on each drive, specify it as /mnt/disk{1...4}/minio.

MinIO does not support arbitrary migration of a drive with existing MinIO data to a new mount position, whether intentional or as the result of OS-level behavior.

Network File System Volumes Break Consistency Guarantees

MinIO’s strict read-after-write and list-after-write consistency model requires local drive filesystems.

MinIO cannot provide consistency guarantees if the underlying storage volumes are NFS or a similar network-attached storage volume.

For deployments that require using network-attached storage, use NFSv4 for best results.

Deploy Single-Node Multi-Drive MinIO

The following procedure deploys MinIO consisting of a single MinIO server and a multiple drives or storage volumes.

1) Download the MinIO Server

The following tabs provide examples of installing MinIO onto 64-bit Linux operating systems using RPM, DEB, or binary. The RPM and DEB packages automatically install MinIO to the necessary system paths and create a systemd service file for running MinIO automatically. MinIO strongly recommends using RPM or DEB installation routes.

Use the following commands to download the latest stable MinIO RPM and install it.

wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio-20221207005637.0.0.x86_64.rpm -O minio.rpm
sudo dnf install minio.rpm

Use the following commands to download the latest stable MinIO DEB and install it:

wget https://dl.min.io/server/minio/release/linux-amd64/archive/minio_20221207005637.0.0_amd64.deb -O minio.deb
sudo dpkg -i minio.deb

Use the following commands to download the latest stable MinIO binary and install it to the system $PATH:

wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
sudo mv minio /usr/local/bin/

2) Create the systemd Service File

The .deb or .rpm packages install the following systemd service file to /etc/systemd/system/minio.service. For binary installations, create this file manually on all MinIO hosts:

[Unit]
Description=MinIO
Documentation=https://min.io/docs/minio/linux/index.html
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio

[Service]
WorkingDirectory=/usr/local

User=minio-user
Group=minio-user
ProtectProc=invisible

EnvironmentFile=-/etc/default/minio
ExecStartPre=/bin/bash -c "if [ -z \"${MINIO_VOLUMES}\" ]; then echo \"Variable MINIO_VOLUMES not set in /etc/default/minio\"; exit 1; fi"
ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES

# Let systemd restart this service always
Restart=always

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of threads this process can create
TasksMax=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=multi-user.target

# Built for ${project.name}-${project.version} (${project.name})

The minio.service file runs as the minio-user User and Group by default. You can create the user and group using the groupadd and useradd commands. The following example creates the user, group, and sets permissions to access the folder paths intended for use by MinIO. These commands typically require root (sudo) permissions.

groupadd -r minio-user
useradd -M -r -g minio-user minio-user
chown minio-user:minio-user /mnt/disk1 /mnt/disk2 /mnt/disk3 /mnt/disk4

The specified drive paths are provided as an example. Change them to match the path to those drives intended for use by MinIO.

Alternatively, change the User and Group values to another user and group on the system host with the necessary access and permissions.

MinIO publishes additional startup script examples on github.com/minio/minio-service.

3) Create the Environment Variable File

Create an environment variable file at /etc/default/minio. For Windows hosts, specify a Windows-style path similar to C:\minio\config. The MinIO Server container can use this file as the source of all environment variables.

The following example provides a starting environment file:

# MINIO_ROOT_USER and MINIO_ROOT_PASSWORD sets the root account for the MinIO server.
# This user has unrestricted permissions to perform S3 and administrative API operations on any resource in the deployment.
# Omit to use the default values 'minioadmin:minioadmin'.
# MinIO recommends setting non-default values as a best practice, regardless of environment.

MINIO_ROOT_USER=myminioadmin
MINIO_ROOT_PASSWORD=minio-secret-key-change-me

# MINIO_VOLUMES sets the storage volumes or paths to use for the MinIO server.
# The specified path uses MinIO expansion notation to denote a sequential series of drives between 1 and 4, inclusive.
# All drives or paths included in the expanded drive list must exist *and* be empty or freshly formatted for MinIO to start successfully.

MINIO_VOLUMES="/data-{1...4}"

# MINIO_SERVER_URL sets the hostname of the local machine for use with the MinIO Server.
# MinIO assumes your network control plane can correctly resolve this hostname to the local machine.

# Uncomment the following line and replace the value with the correct hostname for the local machine.

#MINIO_SERVER_URL="http://minio.example.net"

Include any other environment variables as required for your local deployment. ..

4) Start the MinIO Service

Issue the following command on the local host to start the MinIO SNSD deployment as a service:

sudo systemctl start minio.service

Use the following commands to confirm the service is online and functional:

sudo systemctl status minio.service
journalctl -f -u minio.service

MinIO may log an increased number of non-critical warnings while the server processes connect and synchronize. These warnings are typically transient and should resolve as the deployment comes online.

The journalctl output should resemble the following:

Status:         1 Online, 0 Offline.
API: http://192.168.2.100:9000  http://127.0.0.1:9000
RootUser: myminioadmin
RootPass: minio-secret-key-change-me
Console: http://192.168.2.100:9090 http://127.0.0.1:9090
RootUser: myminioadmin
RootPass: minio-secret-key-change-me

Command-line: https://min.io/docs/minio/linux/reference/minio-mc.html
   $ mc alias set myminio http://10.0.2.100:9000 myminioadmin minio-secret-key-change-me

Documentation: https://min.io/docs/minio/linux/index.html

The API block lists the network interfaces and port on which clients can access the MinIO S3 API. The Console block lists the network interfaces and port on which clients can access the MinIO Web Console.

5) Connect to the MinIO Service

You can access the MinIO Console by entering any of the hostnames or IP addresses from the MinIO server Console block in your preferred browser, such as http://localhost:9090.

Log in with the MINIO_ROOT_USER and MINIO_ROOT_PASSWORD configured in the environment file specified to the container.

MinIO Console displaying Buckets view in a fresh installation

You can use the MinIO Console for general administration tasks like Identity and Access Management, Metrics and Log Monitoring, or Server Configuration. Each MinIO server includes its own embedded MinIO Console.

If your local host firewall permits external access to the Console port, other hosts on the same network can access the Console using the IP or hostname for your local host.

You can access the MinIO deployment over a Terminal or Shell using the MinIO Client (mc). See MinIO Client Installation Quickstart for instructions on installing mc.

Create a new alias corresponding to the MinIO deployment. Specify any of the hostnames or IP addresses from the MinIO Server API block, such as http://localhost:9000.

mc alias set http://localhost:9000 myminioadmin minio-secret-key-change-me

Replace myminioadmin and minio-secret-key-change-me with the MINIO_ROOT_USER and MINIO_ROOT_PASSWORD values in the environment file specified to the container.

You can then interact with the container using any mc command. If your local host firewall permits external access to the MinIO S3 API port, other hosts on the same network can access the MinIO deployment using the IP or hostname for your local host.