Site Replication Overview

1. Policies

2. User accounts (for local users)

3. Groups

4. Access Keys

   Access Keys for root do not sync.

5. Policy mapping for synced user accounts

6. Policy mapping for Security Token Service (STS) users

1. Policies

2. Access Keys associated to OIDC accounts with a valid MinIO Policy. root access keys do not sync.

3. Policy mapping for synced user accounts

4. Policy mapping for Security Token Service (STS) users

1. Policies

2. Groups

3. Access Keys associated to LDAP accounts with a valid MinIO Policy. root access keys do not sync.

4. Policy mapping for synced user accounts

5. Policy mapping for Security Token Service (STS) users

1. Deploy two or more separate MinIO sites, using the same Identity Provider for each site

   Only one site can have any buckets or objects on it. The other site(s) must be empty.

2. In a browser, access the Console for the site with data (if any)

   For example, https://<addressforsite>:9000

   Replace <addressforsite> with the hostname or IP address of the load balancer, reverse proxy, or similar control plane that manages connections to the MinIO deployment.

3. Select Settings, then Site Replication

4. Select Add Sites +

5. Complete the requested information:

   This Site:

   • Site Name:

     A name or other identifying text to associate to the site.

   • Endpoint:

     (required) The hostname or IP of the load balancer managing connections to the site.

   • Access Key:

     (required) The user name for root to use for signing in to the site.

   • Secret Key:

     (required) The password for root to use for signing in to the site.

   Peer Sites:

   • Site Name:

     A name or other identifying text to associate to the site.

   • Endpoint:

     (required) The hostname or IP of the load balancer managing connections to the site.

   • Access Key:

     (required) The user name for root for the peer site to use for signing in to the site.

   • Secret Key:

     (required) The password for root for the peer site to use for signing in to the site.

   Specify the URL or IP address of the site’s load balancer, reverse proxy, or similar network control plane component. Requests are automatically routed to nodes in the deployment.

   MinIO recommends against using a single node hostname for a peer site. This creates a single point of failure: if that node goes offline, replication fails.

   To add additional sites beyond two, select the + button to the side of one of the Site entries. To remove a site previously added, select the - button to the side of the site.

   Site replication adds a svcacct under the root user of each site to perform replication activities.

6. Select Save

7. Select Replication Status button to verify replication has completed across peer sites.

   Any replicable data that exists should show as successfully synced.

   For more on reviewing site replication, see the Site Replication Status tutorial.

The following steps create a new site replication configuration for three distributed deployments. One of the sites contains replicable data.

The three sites use aliases, minio1, minio2, and minio3, and only minio1 contains any data.

1. Deploy three or more separate MinIO sites, using the same IDP

   Start with empty sites or have no more than one site with any replicable data.

2. Configure an alias for each site

   Specify the URL or IP address of the site’s load balancer, reverse proxy, or similar network control plane component. Requests are automatically routed to nodes in the deployment.

   MinIO recommends against using a single node hostname for a peer site. This creates a single point of failure: if that node goes offline, replication fails.

   For example, for three MinIO sites, you might create aliases minio1, minio2, and minio3.

   Use mc alias set to define the hostname or IP of the load balancer managing connections to the site.

   mc alias set minio1 https://minio1.example.com:9000 adminuser adminpassword
   mc alias set minio2 https://minio2.example.com:9000 adminuser adminpassword
   mc alias set minio3 https://minio3.example.com:9000 adminuser adminpassword
   

   or define environment variables

   export MC_HOST_minio1=https://adminuser:adminpassword@minio1.example.com
   export MC_HOST_minio2=https://adminuser:adminpassword@minio2.example.com
   export MC_HOST_minio3=https://adminuser:adminpassword@minio3.example.com
   

3. Add site replication configuration

   mc admin replicate add minio1 minio2 minio3
   

   If all sites are empty, the order of the aliases does not matter. If one of the sites contains any replicable data, you must list it first.

   No more than one site can contain any replicable data.

4. Query the site replication configuration to verify

   mc admin replicate info minio1
   

   You can use the alias for any peer site in the site replication configuration.

5. Query the site replication status to confirm any initial data has replicated to all peer sites.

   mc admin replicate status minio1
   

   You can use the alias for any of the peer sites in the site replication configuration. The output should say that all replicable data is in sync.

   The output could resemble the following:

   Bucket replication status:
   ●  1/1 Buckets in sync
   
   Policy replication status:
   ●  5/5 Policies in sync
   
   User replication status:
   No Users present
   
   Group replication status:
   No Groups present
   

   For more on reviewing site replication, see the Site Replication Status tutorial.

1. Deploy a new, empty MinIO site

2. In a browser, access the Console for one of the existing replicated sites

   For example, https://<addressforsite>:9000

3. Select Settings, then Site Replication

4. Select Add Sites +

5. Make the following entries:

   Access Key:

   (required) The user name to use for signing in to each site. Should be the same across all sites.

   Secret Key:

   (required) The password for the user name to use for signing in to each site. Should be the same across all sites.

   Site Name:

   An alias to use for the site name.

   Endpoint:

   (required) The hostname or IP of the load balancer managing connections to the site.

   Specify the URL or IP address of the site’s load balancer, reverse proxy, or similar network control plane component. Requests are automatically routed to nodes in the deployment.

   MinIO recommends against using a single node hostname for a peer site. This creates a single point of failure: if that node goes offline, replication fails.

   To add additional sites beyond two, select the + button to the side of the last Site entry.

6. Select Save

1. Deploy the new MinIO peer site(s) following the stated requirements

2. Configure an alias for the new site

   Specify the URL or IP address of the site’s load balancer, reverse proxy, or similar network control plane component. Requests are automatically routed to nodes in the deployment.

   MinIO recommends against using a single node hostname for a peer site. This creates a single point of failure: if that node goes offline, replication fails.

   To check the existing aliases, use mc alias list.

   Use mc alias set to define the hostname or IP of the load balancer managing connections to the new site(s).

   mc alias set minio4 https://minio4.example.com:9000 adminuser adminpassword
   

   or define environment variables

   export MC_HOST_minio4=https://adminuser:adminpassword@minio4.example.com
   

3. Add site replication configuration

   Use the mc admin replicate add command to expand the site replication configuration with the new peer site. Specify the alias of all existing peer sites, then the alias of the new site to add.

   For example, the following command adds the new peer site minio4 to an existing site replication configuration that includes the existing sites minio1, minio2, and minio3.

   mc admin replicate add minio1 minio2 minio3 minio4
   

   Note

   If any of the sites are unreachable or permanently lost, you must first remove the unreachable site(s) with mc admin replicate rm before expanding with the new site.

4. Query the site replication configuration to verify

   mc admin replicate info minio1
   

1. In a browser, access the Console for one of the replicated sites

   For example, https://<addressforsite>:9000

2. Select Settings, then Site Replication

3. Select the pencil Edit icon to the side of the site to update

4. Make the following entries:

   New Endpoint:

   (required) The new endpoint address and port to use.

   Specify the URL or IP address of the site’s load balancer, reverse proxy, or similar network control plane component. Requests are automatically routed to nodes in the deployment.

   MinIO recommends against using a single node hostname for a peer site. This creates a single point of failure: if that node goes offline, replication fails.

5. Select Update

1. Obtain the site’s Deployment ID with mc admin replicate info

   mc admin replicate info <ALIAS>
   

2. Update the site’s endpoint with mc admin replicate update

   mc admin replicate update ALIAS --deployment-id [DEPLOYMENT-ID] --endpoint [NEW-ENDPOINT]
   

   Replace [DEPLOYMENT-ID] with the deployment ID of the site to update.

   Replace [NEW-ENDPOINT] with the new endpoint for the site.

   Specify the URL or IP address of the site’s load balancer, reverse proxy, or similar network control plane component. Requests are automatically routed to nodes in the deployment.

   MinIO recommends against using a single node hostname for a peer site. This creates a single point of failure: if that node goes offline, replication fails.

1. In a browser, access the Console for one of the replicated sites

   For example, https://<addressforsite>:9000

2. Select Settings, then Site Replication

3. Select the trash can Delete icon to the side of the site to update

4. Confirm the site deletion at the prompt by selecting Delete

Use mc admin replicate rm

mc admin replicate rm ALIAS PEER_TO_REMOVE --force

• Replace ALIAS with the alias of any peer site in the replication configuration.

• Replace PEER_TO_REMOVE with the alias of the peer site to remove.

All healthy peers in the site replication configuration update to remove the specified peer automatically.

MinIO requires the --force flag to remove the peer from the site replication configuration.

1. In a browser, access the Console for one of the replicated sites

   For example, https://<addressforsite>:9000

2. Select Settings, then Site Replication

3. Select Replication Status

4. (Optional) View the replication status for a specific item

   Select the type of item to view in the View Replication Status for a: dropdown

   Specify the name of the specific Bucket, Group, Policy, or User to view

5. (Optional) Update the information by selecting Refresh

Use mc admin replicate status

mc admin replicate status <ALIAS> --<flag> <value>

For example:

• mc admin replicate status minio3 --bucket images

  Displays the replication status for the images bucket on the minio3 site.

  The output resembles the following:

  ●  Bucket config replication summary for: images
  
  Bucket          | MINIO2          | MINIO3          | MINIO4
  Tags            |                 |                 |
  Policy          |                 |                 |
  Quota           |                 |                 |
  Retention       |                 |                 |
  Encryption      |                 |                 |
  Replication     | ✔               | ✔               | ✔
  

• mc admin replicate status minio3 --all

  Displays the replication status summary for all replication sites of which minio3 is part.

  The output resembles the following:

  Bucket replication status:
  ●  1/1 Buckets in sync
  
  Policy replication status:
  ●  5/5 Policies in sync
  
  User replication status:
  ●  1/1 Users in sync
  
  Group replication status:
  ●  0/2 Groups in sync
  
  Group           | MINIO2          | MINIO3          | MINIO4
  ittechs         | ✗  in-sync      |                 | ✗  in-sync
  managers        | ✗  in-sync      |                 | ✗  in-sync