Configure disk storage

Wed, 08 Apr 2026 14:38:28 +0000

Configure disk storage

The ingester component in Grafana Pyroscope processes the received profiling data. First it keeps the data organized in memory, in the so-called “head block”. Once the size of the head block exceeds a threshold or the head block is older than -pyroscopedb.max-block-duration (by default 1 hour), the ingester writes the block to the local persistent disk. Refer to block format for more detail about the block’s layout.

Each of those blocks are identified by an ULID and stored within Pyroscope’s data path -pyroscopedb.data-path= (by default ./data). This directory is organized by the following:

./<tenant-id>: Each tenant has its own subdirectory with the following subdirectories:
- head/<block-id>: Contains the current data still being written.
- local/<block-id>: Contains the finished blocks, which are kept locally

Object storage

When object storage is configured, finished blocks are uploaded to the object store bucket.

High disk utilization

To avoid losing the most recent data, Pyroscope removes the oldest blocks when it detects that the volume on which the data path is located is close to running out of disk space. The check for high disk utilization runs every -pyroscopedb.retention-policy-enforcement-interval when:

less than -pyroscopedb.retention-policy-min-disk-available-percentage=0.05 of the total size of the volume is available and
the available disk space smaller then -pyroscopedb.retention-policy-min-free-disk-gb=10.

The deletion is logged like this:

level=warn caller=pyroscopedb.go:231 ts=2022-10-05T13:19:09.770693308Z msg="disk utilization is high, deleted oldest block" path=data/anonymous/local/01GDZYHKKKY2ANY6PCJJZGT1N8

Configure object storage backend

Wed, 08 Apr 2026 14:38:28 +0000

Configure object storage backend

Grafana Pyroscope can use different object storage services to persist blocks containing the profiles data. Blocks are flushed by ingesters on disk first then are uploaded to the object store.

The supported backends are:

Amazon S3 and compatible implementations like MinIO
Google Cloud Storage
Azure Blob Storage
Swift (OpenStack Object Storage)

Internally, Pyroscope uses Thanos’ object store client library, so their stated limitations apply.

Amazon S3

To use an AWS S3 or S3-compatible bucket for long term storage, you can find Pyroscope’s configuration parameters in the reference config. Apart from those parameters, it is also possible to supply configuration parameters using the well-known environment variables of the AWS SDK.

At a minimum, you will need to provide values for the bucket_name, endpoint, access_key_id, and secret_access_key keys.

Example using an AWS Bucket

This how one would configure a bucket in the AWS region eu-west-2:

storage:
  backend: s3
  s3:
    bucket_name: #REPLACE_WITH_BUCKET_NAME
    region: eu-west-2
    endpoint: s3.eu-west-2.amazonaws.com
    access_key_id: #REPLACE_WITH_ACCESS_KEY
    secret_access_key: #REPLACE_WITH_SECRET_KEY

Example using a S3 compatible Bucket

This how one would configure a bucket on a locally running instance of MinIO:

storage:
  backend: s3
  s3:
    bucket_name: grafana-pyroscope-data
    endpoint: localhost:9000
    insecure: true
    access_key_id: grafana-pyroscope-data
    secret_access_key: grafana-pyroscope-data

Google Cloud Storage

To use a Google Cloud Storage (GCS) bucket for long term storage, you can find Pyroscope’s configuration parameters in the reference config.

At a minimum, you will need to provide a values for the bucket_name and a service account. To supply the service account there are two ways:

Use the GOOGLE_APPLICATION_CREDENTIALS environment variable to locate your application credentials.
Provide the content the service account key within the service_account parameter.

Example using a Google Cloud Storage bucket

This how one would configure a GCS bucket using the service_account parameter:

storage:
  backend: gcs
  gcs:
    bucket_name: grafana-pyroscope-data
    service_account: |
      {
        "type": "service_account",
        "project_id": "PROJECT_ID",
        "private_key_id": "KEY_ID",
        "private_key": "-----BEGIN PRIVATE KEY-----\nPRIVATE_KEY\n-----END PRIVATE KEY-----\n",
        "client_email": "SERVICE_ACCOUNT_EMAIL",
        "client_id": "CLIENT_ID",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://accounts.google.com/o/oauth2/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/SERVICE_ACCOUNT_EMAIL"
      }

Azure Blob Storage

To use an Azure Blob Storage bucket for long term storage, you can find Pyroscope’s configuration parameters in the reference config.

If user_assigned_id is used, authentication is done via user-assigned managed identity.

Swift (OpenStack Object Storage)

To use a Swift (OpenStack Object Storage) bucket for long term storage, you can find Pyroscope’s configuration parameters in the reference config.

If the name of a user, project or tenant is used one must also specify its domain by ID or name. Various examples for OpenStack authentication can be found in the official documentation.

Configure storage on Grafana Labs

Configure disk storage

Configure disk storage

Object storage

High disk utilization

Configure object storage backend

Configure object storage backend

Amazon S3

Example using an AWS Bucket

Example using a S3 compatible Bucket

Google Cloud Storage

Example using a Google Cloud Storage bucket

Azure Blob Storage

Swift (OpenStack Object Storage)