mc pipe
Table of Contents
Syntax
The mc pipe
command streams content from STDIN to a target object.
The following command writes contents of STDIN
to an S3 compatible storage.
echo "My Meeting Notes" | mc pipe s3/engineering/meeting-notes.txt
The command has the following syntax:
mc [GLOBALFLAGS] pipe \
TARGET \
[--encrypt "string"] \
[--storage-class, --sc "string"] \
[--attr "string"] \
[--tags "string"] \
[--encrypt-key "string"]
Brackets
[]
indicate optional parameters.Parameters sharing a line are mutually dependent.
Parameters seperated using the pipe
|
operator are mutually exclusive.
Copy the example to a text editor and modify as-needed before running the command in the terminal/shell.
Changed in version RELEASE.2023-01-11T03-14-16Z: mc pipe
now supports concurrent uploads for better throughput of large streams.
Parameters
- TARGET
- Required
The full path to the alias or prefix where the command should run.
- --attr
- Optional
Add custom metadata for the object.
Specify key-value pairs as
KEY=VALUE\;
, separating each pair with a back slash and semicolon (\;
). For example,--attr key1=value1\;key2=value2\;key3=value3
.
- --encrypt
- Optional
Encrypt or decrypt objects using server-side encryption with server-managed keys. Specify key-value pairs as
KEY=VALUE
.Each
KEY
represents a bucket or object.Each
VALUE
represents the data key to use for encrypting object(s).
Enclose the entire list of key-value pairs passed to
--encrypt
in double-quotes"
.--encrypt
can use theMC_ENCRYPT
environment variable for retrieving a list of encryption key-value pairs as an alternative to specifying them on the command line.
- --encrypt-key
- Optional
Encrypt or decrypt objects using server-side encryption with client-specified keys. Specify key-value pairs as
KEY=VALUE
.Each
KEY
represents a bucket or object.Each
VALUE
represents the data key to use for encrypting object(s).
Enclose the entire list of key-value pairs passed to
--encrypt-key
in double quotes"
.--encrypt-key
can use theMC_ENCRYPT_KEY
environment variable for retrieving a list of encryption key-value pairs as an alternative to specifying them on the command line.
- --storage-class, --sc
- Optional
Set the storage class for the new object at the
TARGET
.See Amazons documentation for more information on S3 storage classes.
Global Flags
This command supports any of the global flags.
Examples
Write Contents of STDIN
to the Local Filesystem
The following command writes the contents of STDIN to the /tmp
folder on the local filesystem.
mc pipe /tmp/hello-world.go
Copy an ISO Image to S3 Storage
The following command first streams the contents of an iso image for Debian and then uses the stream to create the object at an S3 path.
cat debian-live-11.5.0-amd64-mate.iso | mc pipe s3/opensource-isos/debian-11-5.iso
Stream MySQL Database Dump to S3
The following command first streams a MySQL database and uses the stream to create a backup on S3 with mc pipe
:
mysqldump -u root -p ******* accountsdb | mc pipe s3/sql-backups/backups/accountsdb-sep-28-2022.sql
Write a File to a Reduced Redundancy Storage Class
The following command takes the STDIN stream and creates an object on the Reduced Redundancy storage class on S3.
mc pipe --storage-class REDUCED_REDUNDANCY s3/personalbuck/meeting-notes.txt
Copy a File to a MinIO Deployment with Metadata
The following command uploads an MP3 file to a MinIO deployment with an ALIAS of myminio
and a music
bucket.
The object writes with some metadata for Cache-Control
and Artist
.
cat music.mp3 | mc pipe --attr "Cache-Control=max-age=90000,min-fresh=9000;Artist=Unknown" myminio/music/guitar.mp3