Object Storage Monitoring

Metrics and logging are critical when tracking the health and performance of any system. MinIO provides complete visibility into clusters with detailed storage performance monitoring, metrics, and per-operation logging. The result is a robust, transparent and performant answer to object storage monitoring, alerting and observability.

Metrics

MinIO exports a wide range of granular hardware and software metrics through a Prometheus-compatible metrics endpoint. Prometheus is a cloud-native monitoring platform consisting of a multi-dimensional data model with time series data identified by metric name and key/value pairs. MinIO includes a storage monitoring dashboard using Grafana to visualize collected metrics. The Prometheus ecosystem includes multiple integrations for routing MinIO metrics to storage, messaging and alert services.

MinIO surfaces a wide range of granular hardware and software metrics through the Prometheus endpoint, including health information such as disk or node failure, total free storage capacity and per-disk storage capacity. Leveraging Prometheus and its growing adoption as a leading metrics collection and analysis platform allows MinIO to focus on its object storage capabilities, instead of building out innumerable custom data storage monitoring adapters for a given third-party analysis/visualization/alerting service.

The MinIO Kubernetes Operator can automatically deploy, configure and manage Prometheus deployments and metrics collection per tenant. Organizations can also point their own Prometheus or Prometheus-compatible systems at each tenant for centralized monitoring across multiple vendors, data centers and visualization/analytics tools.

MinIO also provides a healthcheck endpoint for probing node and cluster liveness. A simple CURL statement can indicate whether a given node is healthy or whether a cluster has read/write quorum.

MinIO's Object Storage monitoring dashboard

Logs

Enabling MinIO audit logging directs MinIO to generate a log for every operation on the cluster. Every operation generates an audit log with a unique ID and detailed information on the client, object, bucket and all other operation-related metadata. MinIO writes log data to a configured HTTP/HTTPS webhook endpoint. Custom adapters are available to meet specific requirements for audit logging targets.

MinIO supports configuring audit logs through both the MinIO Console UI and the MinIO `mc` command line tool. For Kubernetes environments, the MinIO Operator automatically configures the Console with a LogSearch integration for visual inspection of collected audit logs.

MinIO Lambda Notifications provide additional logging support. MinIO can automatically send bucket and object events to third-party applications for event-driven processing such as serverless or function-as-a-service computing frameworks. MinIO Lambda Notifications support targets such as RabbitMQ, Kafka, Elasticsearch, and arbitrary services via webhook.

MinIO also supports live tracing of HTTP/S operations through the MinIO Console and MinIO mc admin trace shell command.

Architecture

MinIO exposes it's metrics through a Prometheus-compatible HTTP(S) endpoint, where the Prometheus service provides push/pull access to those metrics. The MinIO Kubernetes Operator deploys an independent Prometheus service per MinIO Tenant preconfigured to scrape tenant metrics. Organizations can also deploy or utilize their own centralized Prometheus service for scraping tenant metrics.

Prometheus-compatible performance monitoring for object storage

MinIO Lambda Notifications automatically push event notifications to supported target services, such as Kafka, Elasticsearch, or PostgreSQL. Administrators can define bucket-level notification rules with granular filters around which S3 events and objects for which MinIO generates events. MinIO Lambda Notifications are built into the MinIO object storage service and only require access to the remote notification target.

MinIO Tenant event monitoring
Requirements
For Metrics
BYO Prometheus *or* use Kubernetes Operator to auto-deploy/configure per tenant.
For Logs
Supported third-party targets for notifications.
For Log Search
BYO PostgreSQL *or* use Kubernetes Operator to auto-deploy/configure per tenant.

Learn more about monitoring your MinIO instances

Monitor MinIO server with Prometheus
Blog
Monitor MinIO server with Prometheus
Read Story
Using Splunk to Monitor MinIO - A Tutorial
Blog
Using Splunk to Monitor MinIO - A Tutorial
Read Story
Object Storage as a Service with Kubernetes and MinIO's Operator
Blog
Object Storage as a Service with Kubernetes and MinIO's Operator
Read Story

You are using Internet Explorer version 11 or lower. Due to security issues and lack of support for web standards, it is highly recommended that you upgrade to a modern browser.