--- title: "Amazon S3 and S3-compatible storage | Grafana Tempo documentation" description: "Configure Tempo with Amazon S3 or S3-compatible object stores such as MinIO, SeaweedFS, or rclone." --- > For a curated documentation index, see [llms.txt](/llms.txt). For the complete documentation index, see [llms-full.txt](/llms-full.txt). # Amazon S3 and S3-compatible storage Tempo supports Amazon S3 and S3-compatible object stores as backends for trace storage. For general storage configuration options, refer to the storage section on the [configuration](/docs/tempo/next/configuration/#storage) page. ## Authentication The following authentication methods are supported: - AWS environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` - Static access key and secret credentials specified in `access_key` and `secret_key` - MinIO environment variables `MINIO_ACCESS_KEY` and `MINIO_SECRET_KEY` - AWS shared credentials [configuration file](https://docs.aws.amazon.com/ses/latest/DeveloperGuide/create-shared-credentials-file.html) - MinIO client credentials [configuration file](https://docs.min.io/enterprise/aistor-object-store/reference/cli/#configuration-file) - AWS IAM ([IRSA via WebIdentity](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html), - AWS [EC2 instance role](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)) - AWS [EKS Pod Identity](https://docs.aws.amazon.com/eks/latest/userguide/pod-identities.html) ### IAM policy The following IAM policy shows minimal permissions required by Tempo, where the bucket has already been created. JSON ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "TempoPermissions", "Effect": "Allow", "Action": [ "s3:PutObject", "s3:GetObject", "s3:ListBucket", "s3:DeleteObject", "s3:GetObjectTagging", "s3:PutObjectTagging" ], "Resource": ["arn:aws:s3:::/*", "arn:aws:s3:::"] } ] } ``` ### Lifecycle policy A lifecycle policy is recommended that deletes incomplete multipart uploads after one day. ## S3-compatible local stores for testing > Note > > The tools in this section are provided for local testing and evaluation only. They have not been fully tested with Tempo and are not recommended for production use. You can run an S3-compatible object store locally to test Tempo with the `s3` storage backend. [SeaweedFS](https://github.com/seaweedfs/seaweedfs) is the recommended option for local testing, with a single-command startup and a built-in web UI. [rclone serve s3](https://rclone.org/commands/rclone_serve_s3/) is an alternative that serves any local directory as an S3-compatible endpoint. It is classified as experimental by the `rclone` project and has known limitations. [MinIO](https://min.io/) is also supported but its [open source repository has been archived](https://github.com/minio/minio) and the community edition is now distributed as source code only. Pre-compiled binaries are no longer published. ### Set up a local S3-compatible object store Choose a tab below to set up your preferred object store: SeaweedFSrclone serve s3 (experimental)MinIO SeaweedFS rclone serve s3 (experimental) MinIO > Note > > SeaweedFS has not been fully tested with Tempo and is provided here as an alternative for local evaluation only. It isn’t recommended for production use with Tempo. [SeaweedFS](https://github.com/seaweedfs/seaweedfs) is an Apache 2.0-licensed distributed storage system with a built-in S3 gateway. 1. Download and install SeaweedFS from the [releases page](https://github.com/seaweedfs/seaweedfs/releases). 2. Create a data directory and start SeaweedFS in mini mode: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash sudo mkdir -p /data/seaweedfs sudo chown -R $USER:$USER /data/seaweedfs weed mini -dir=/data/seaweedfs ``` The `weed mini` command starts a complete single-node setup including the S3 gateway on port 8333. SeaweedFS runs in the foreground, so open a new terminal for the remaining steps. 3. Create a bucket called `tempo` using the AWS CLI: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash aws --endpoint-url http://localhost:8333 s3 mb s3://tempo --no-sign-request ``` You need the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) installed. SeaweedFS mini mode allows anonymous access, so the `--no-sign-request` flag skips credential checks. > Note > > `rclone serve s3` is classified as experimental by the rclone project and hasn’t been fully tested with Tempo. It’s provided as an alternative for local evaluation only and isn’t recommended for production use. Refer to the [rclone documentation](https://rclone.org/commands/rclone_serve_s3/) for current limitations. [rclone](https://rclone.org/) can serve any local directory as an S3-compatible endpoint. 1. Install rclone by following the [rclone install guide](https://rclone.org/install/). 2. Create a data directory and start the S3 server: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash sudo mkdir -p /data/rclone-s3 sudo chown -R $USER:$USER /data/rclone-s3 rclone serve s3 /data/rclone-s3 --auth-key tempokey,temposecret --addr :8080 ``` The server runs in the foreground on port 8080. Open a new terminal for the remaining steps. 3. Create a bucket called `tempo` using the AWS CLI: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash AWS_ACCESS_KEY_ID=tempokey AWS_SECRET_ACCESS_KEY=temposecret \ aws --endpoint-url http://localhost:8080 s3 mb s3://tempo ``` You need the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) installed. Use the credentials you set with the `--auth-key` flag. > Note > > The MinIO open source repository has been archived and the community edition is now source-only. Pre-compiled binaries are no longer published. You must build MinIO from source using Go 1.24 or later. 1. Install MinIO from source: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash go install github.com/minio/minio@latest ``` Refer to the [MinIO repository](https://github.com/minio/minio) for alternative installation methods including building a Docker image. 2. Create a data directory and start MinIO: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash sudo mkdir -p /data/minio sudo chown -R $USER:$USER /data/minio minio server /data/minio --console-address ':9001' ``` By default, MinIO uses `minioadmin` for both the access key and secret key. MinIO runs in the foreground, so open a new terminal for the remaining steps. 3. Create a bucket called `tempo` using the MinIO Client (`mc`): Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash mc alias set local http://localhost:9000 minioadmin minioadmin mc mb local/tempo ``` ### Tempo configuration for S3-compatible stores The following example configuration uses the S3 backend. Replace the ``, ``, and `` placeholders with the values for your object store. This example configuration includes the metrics-generator. To disable it, remove the `metrics_generator` block and the `processors` list from the overrides. YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```yaml stream_over_http_enabled: true server: http_listen_port: 3200 distributor: receivers: otlp: protocols: grpc: endpoint: "0.0.0.0:4317" http: endpoint: "0.0.0.0:4318" backend_scheduler: provider: compaction: compaction: block_retention: 1h backend_worker: compaction: block_retention: 1h metrics_generator: registry: external_labels: source: tempo cluster: linux-monolithic storage: path: /tmp/tempo/generator/wal remote_write: - url: http:///api/v1/write send_exemplars: true storage: trace: backend: s3 s3: endpoint: bucket: tempo access_key: secret_key: insecure: true wal: path: /var/tempo/wal overrides: defaults: metrics_generator: processors: [service-graphs, span-metrics] usage_report: reporting_enabled: false ``` Replace the `` placeholder with the address of your Prometheus-compatible storage instance (for example, `localhost:9090`). To disable the metrics-generator, remove the `processors` list from the overrides and the `metrics_generator` block. Use the following endpoint and credential values for each object store: SeaweedFSrclone serve s3 (experimental)MinIO SeaweedFS rclone serve s3 (experimental) MinIO SeaweedFS mini mode allows anonymous access, so the `access_key` and `secret_key` fields can be omitted or set to any value: YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```yaml storage: trace: backend: s3 s3: endpoint: localhost:8333 bucket: tempo insecure: true ``` Use the credentials you set with the `--auth-key` flag when starting `rclone`: YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```yaml storage: trace: backend: s3 s3: endpoint: localhost:8080 bucket: tempo access_key: tempokey secret_key: temposecret insecure: true ``` YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```yaml storage: trace: backend: s3 s3: endpoint: localhost:9000 bucket: tempo access_key: minioadmin secret_key: minioadmin insecure: true ``` ### Verify data in your S3-compatible store After traces start flowing, verify that your storage bucket has received data: SeaweedFSrclone serve s3 (experimental)MinIO SeaweedFS rclone serve s3 (experimental) MinIO Open the SeaweedFS admin UI at `http://localhost:23646`, or list the bucket contents using the AWS CLI: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash aws --endpoint-url http://localhost:8333 s3 ls s3://tempo/ --recursive --no-sign-request ``` You should see files such as `single-tenant//data.parquet` and `single-tenant//meta.json`. List the bucket contents using the AWS CLI: Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy ```bash AWS_ACCESS_KEY_ID=tempokey AWS_SECRET_ACCESS_KEY=temposecret \ aws --endpoint-url http://localhost:8080 s3 ls s3://tempo/ --recursive ``` You should see files such as `single-tenant//data.parquet` and `single-tenant//meta.json`. `rclone serve s3` does not provide a web UI. Open the MinIO Console at `http://localhost:9001` and check the `tempo` bucket for files such as `work.json` and a tenant data directory.