mc batch generate

Parameters

TARGET

Required

The alias used to generate the YAML template file. The specified alias does not restrict the deployment(s) where you can use the generated file.

For example:

mc batch generate myminio replicate

JOBTYPE

Required

The type of job to generate a YAML document for.

Currently, mc batch only supports the replicate job type.

Global Flags

This command supports any of the global flags.

Generate a yaml File for a Replicate Job Type

The following command generates a YAML blueprint for a replicate type batch job and names the file replicate with the .yaml extension:

mc batch generate alias replicate > replicate.yaml

• Replace alias with the alias to use to generate the yaml file.

• Replace replicate with the type of job to generate a yaml file for.

  At the time of release, :mc:mc batch only supports the replicate job type.

S3 Compatibility

The mc commandline tool is built for compatibility with the AWS S3 API and is tested MinIO and AWS S3 for expected functionality and behavior.

MinIO provides no guarantees for other S3-compatible services, as their S3 API implementation is unknown and therefore unsupported. While mc commands may work as documented, any such usage is at your own risk.

replicate

Use the replicate job type to create a batch job that replicates objects from the local MinIO deployment to another MinIO location.

The YAML must define the source and target deployments. Optionally, the YAML can also define flags to filter which objects replicate, send notifications for the job, or define retry attempts for the job.

For the source deployment

• Required information

  type:	Must be minio.
  bucket:	The bucket on the deployment.

• Optional information

  prefix:	The prefix on the object(s) that should replicate.
  endpoint:	Location of the source deployment, must be local.
  credentials:	The accesskey: and secretKey: or the sessionToken: that grants access to the object(s).

For the target deployment

• Required information

  type:	Must be minio.
  bucket:	The bucket on the deployment.

• Optional information

  prefix:	The prefix on the object(s) to replicate.
  endpoint:	The location of the source deployment. If the location is not remote, use local.
  credentials:	The accesskey and secretKey or the sessionToken that grants access to the object(s).

For filters

For notifications

For retry attempts

If something interrupts the job, you can define how many attempts to retry the job batch. For each retry, you can also define how long to wait between attempts.

replicate:
  apiVersion: v1
  # source of the objects to be replicated
  source:
    type: TYPE # valid values are "s3"
    bucket: BUCKET
    prefix: PREFIX
    # endpoint: ENDPOINT
    # credentials:
    #   accessKey: ACCESS-KEY
    #   secretKey: SECRET-KEY
    #   sessionToken: SESSION-TOKEN # Available when rotating credentials are used

  # target where the objects must be replicated
  target:
    type: TYPE # valid values are "s3"
    bucket: BUCKET
    prefix: PREFIX
    # endpoint: ENDPOINT
    # credentials:
    #   accessKey: ACCESS-KEY
    #   secretKey: SECRET-KEY
    #   sessionToken: SESSION-TOKEN # Available when rotating credentials are used

  # optional flags based filtering criteria
  # for all source objects
  flags:
    filter:
      newerThan: "7d" # match objects newer than this value (e.g. 7d10h31s)
      olderThan: "7d" # match objects older than this value (e.g. 7d10h31s)
      createdAfter: "date" # match objects created after "date"
      createdBefore: "date" # match objects created before "date"

      # tags:
      #   - key: "name"
      #     value: "pick*" # match objects with tag 'name', with all values starting with 'pick'

      ## NOTE: metadata filter not supported when "source" is non MinIO.
      # metadata:
      #   - key: "content-type"
      #     value: "image/*" # match objects with 'content-type', with all values starting with 'image/'

  notify:
    endpoint: "https://notify.endpoint" # notification endpoint to receive job status events
    token: "Bearer xxxxx" # optional authentication token for the notification endpoint

  retry:
    attempts: 10 # number of retries for the job before giving up
    delay: "500ms" # least amount of delay between each retry