Documentation

Documentation

Enable One-Way Server-Side Bucket Replication

The procedure on this page creates a new bucket replication rule for one-way synchronization of objects from one MinIO bucket to another MinIO bucket. The buckets can be on the same MinIO deployment or on separate MinIO deployments.

Active-Passive Replication synchronizes data from a source MinIO deployment to a remote MinIO deployment.

Note

To configure replication between arbitrary S3-compatible services (not necessarily MinIO), use mc mirror.

Requirements

Replication requires all participating clusters meet the following requirements. This procedure assumes you have reviewed and validated those requirements.

For more details, see the Bucket Replication Requirements page.

Considerations

Click to expand any of the following:

Replication of Existing Objects

MinIO supports automatically replicating existing objects in a bucket.

MinIO requires explicitly enabling replication of existing objects using the mc replicate add --replicate or mc replicate update --replicate and including the existing-objects replication feature flag. This procedure includes the required flags for enabling replication of existing objects.

Replication of Delete Operations

MinIO supports replicating S3 DELETE operations onto the target bucket. Specifically, MinIO can replicate versioning Delete Markers and the deletion of specific versioned objects:

  • For delete operations on an object, MinIO replication also creates the delete marker on the target bucket.

  • For delete operations on versions of an object, MinIO replication also deletes those versions on the target bucket.

MinIO requires explicitly enabling replication of delete operations using the mc replicate add --replicate or mc replicate update --replicate. This procedure includes the required flags for enabling replication of delete operations and delete markers.

MinIO does not replicate delete operations resulting from the application of lifecycle management expiration rules.

See Replication of Delete Operations for more complete documentation.

Multi-Site Replication

MinIO supports configuring multiple remote targets per bucket or bucket prefix. For example, you can configure a bucket to replicate data to two or more remote MinIO deployments, where one deployment is a 1:1 copy (replication of all operations including deletions) and another is a full historical record (replication of only non-destructive write operations).

This procedure documents one-way replication to a single remote MinIO deployment. You can repeat this tutorial to replicate a single bucket to multiple remote targets.

Procedure

Configure One-Way Bucket Replication Using the MinIO Console

1) Create a New Bucket Replication Rule

  1. Log in to the MinIO Console for the deployment

  2. Select the Manage button for the bucket to replicate

    After a successful log in, the MinIO Console shows a list of buckets with options to manage or explore each bucket.
  3. Select the Replication section

    After selecting a bucket to manage, MinIO shows summary information about the bucket as well as a navigation list of pages for adjusting the bucket configuration.
  4. Select Add Replication Rule +

  5. Complete the requested information:

    Field

    Description

    Priority

    Enter a number value to indicate the order in which to process replication rules for the bucket. 1 indicates the highest importance.

    Target URL

    The URL of the deployment to replicate data to.

    Use TLS

    Leave the toggle in the ON position if the destination deployment uses TLS. Otherwise, move the toggle to the OFF position.

    Access Key

    The user name to use on the destination deployment. The user must have write access to the bucket to replicate to.

    Secret Key

    The password for the provided Access Key.

    Target Bucket

    The bucket at the destination to write the data to. The target bucket may have the same name as the origin bucket, depending on the destination bucket location.

    Region

    The AWS resource region location of the destination deployment.

    Replication mode

    Leave the default selection of Asynchronous to allow MinIO to replicate data after the write operation completes on the origin ment. Select Synchronous to attempt to complete the replication of the object during its write operation.

    While synchronous replication may result in more reliable synchronization between the origin and destination buckets, it may also increase the time of each write operation.

    Bandwidth

    Specify the maximum amount of bandwidth the replication process can use while replicating data. Enter a number and select a data unit.

    Health Check Duration

    The maximum length of time in seconds MinIO should spend verifying the health of the replicated data on the destination bucket.

    Storage Class

    The class of storage to use on the destination deployment for the replicated data. Valid values are either STANDARD or REDUCED_REDUNDANCY.

    Object Filters

    Limit which objects to replicate from the bucket by Prefix or tags. If you enter multiple tags, the objects must match all tag values.

    Metadata Sync

    Leave selected to also replicate the object’s metadata file. Otherwise, move the toggle to the Off position.

    Delete Markers

    Leave selected to also replicate MinIO’s indication that an object has been deleted and should also be marked deleted at the ation bucket. Otherwise, move the toggle to the Off position to prevent marking the object as deleted in the destination bucket.

    Deletes

    Leave selected to allow replication of the deletion of versions of an object. Otherwise, move the toggle to the Off position to not replicate deletion of object versions.

  6. Select Save to finish adding the replication rule

2) Validate the Replication Configuration

  1. Go to the Buckets section of the MinIO Console

    The default screen when logging into the MinIO Console. The screen shows a list of the buckets available in the Deployment with options to Manage or Browse the bucket contents.
  2. Select the Browse button for the bucket you added replication to

    The contents of a bucket display after selecting to Browse the MinIO bucket. Options including to Rewind, Refresh, or Upload contents.
  3. Select the Upload button to add a new object to the bucket

  4. Select Upload File

  5. Use the interface to add a new object to the bucket

  6. Go to the other deployment’s console and select the destination bucket defined in the replication

Configure One-Way Bucket Replication Using the Command Line (mc)

This procedure uses the aliases SOURCE and REMOTE to reference each MinIO deployment being configured for replication. Replace these values with the appropriate alias for your target MinIO deployments.

This procedure assumes each alias corresponds to a user with the necessary replication permissions.

1) Create a Replication Remote Target

Use the mc admin bucket remote add command to create a replication target from each deployment to the appropriate bucket on the destination deployment. A bucket may have multiple remote targets to different target buckets. No two targets can resolve from one bucket to the same remote bucket.

mc admin bucket remote add ALIAS/BUCKET                    \
   https://RemoteUser:Password@HOSTNAME/BUCKETDESTINATION  \
   --service "replication"
  • Replace ALIAS with the alias of the MinIO deployment that acts as the origin for the replication.

  • Replace BUCKET with the name of the bucket to replicate from on the origin deployment.

  • Replacete RemoteUser with the user name that has the necessary replication permissions

  • Replace Password with the secret key for the RemoteUser.

  • Replace HOSTNAME with the URL of the destination deployment.

  • Replace BUCKETDESTINATION with the name of the bucket to replicate to on the destination deployment.

The command returns an ARN <Amazon Resource Name> similar to the following:

Role ARN = 'arn:minio:replication::<UUID>:BUCKET'

Copy the ARN to use in the next step, noting the MinIO deployment.

2) Create a New Bucket Replication Rule

Use the mc replicate add command to add a new replication rule to each MinIO deployment.

mc replicate add ALIAS/BUCKET \
   --remote-bucket 'arn:minio:replication::<UUID>:DESTINATIONBUCKET' \
   --replicate "delete,delete-marker,existing-objects"
  • Replace ALIAS with the alias of the origin MinIO deployment. The name must match the bucket specified when creating the remote target in the previous step.

  • Replace BUCKET with the name of the bucket to replicate from on the origin deployment.

  • Replace the --remote-bucket value with the ARN for the destination bucket determined in the first step. Ensure you specify the ARN created on the origin deployment. You can use mc admin bucket remote ls to list all remote ARNs configured on the deployment.

  • The --replicate "delete,delete-marker,existing-objects" flag enables the following replication features:

    See mc replicate add --replicate for more complete documentation. Omit any field to disable replication of that component.

Specify any other supported optional arguments for mc replicate add.

3) Validate the Replication Configuration

Use mc cp to copy a new object to the replicated bucket on one of the deployments.

mc cp ~/foo.txt ALIAS/BUCKET

Use mc ls to verify the object exists on the destination bucket:

mc ls ALIAS/BUCKET

See also