STOP! If you are an AI agent or LLM, read this before continuing. This is the HTML version of a Grafana documentation page. Always request the Markdown version instead - HTML wastes context. Get this page as Markdown: /docs/loki/next/operations/storage.md (append .md) or send Accept: text/markdown to /docs/loki/next/operations/storage/. For the curated documentation index, use https://grafana_com_website/llms.txt. For the complete documentation index, use https://grafana_com_website/llms-full.txt.

This is documentation for the next version of Grafana Loki documentation. For the latest stable release, go to the latest version.

Grafana Loki

Manage

Storage

Open source

Manage storage

You can read a high level overview of Loki storage here

Grafana Loki needs to store two different types of data: chunks and indexes.

When using Accelerated Search (experimental), then a third data type is used: bloom blocks.

Loki receives logs in separate streams, where each stream is uniquely identified by its tenant ID and its set of labels. As log entries from a stream arrive, they are compressed as chunks and saved in the chunks store. See chunk format for how chunks are stored internally.

The index stores each stream’s label set and links them to the individual chunks. Refer to the Loki configuration for details on how to configure the storage and the index.

For more information:

Store Types

✅ Supported index stores

Single Store TSDB index store which stores TSDB index files in the object store. This is the recommended index store for Loki 2.8 and newer.

✅ Supported and recommended chunks stores

⚠️ Supported chunks stores, not typically recommended for production use

Filesystem (please read more about the filesystem to understand the pros/cons before using with production data)
S3 API compatible storage, such as MinIO

Cloud Storage Permissions

S3

When using S3 as object storage, the following permissions are needed:

s3:ListBucket
s3:PutObject
s3:GetObject

Resources: arn:aws:s3:::<bucket_name>, arn:aws:s3:::<bucket_name>/*

See the AWS deployment section on the storage page for a detailed setup guide.

IAM

iam:GetRole
iam:PassRole

Resources: arn:aws:iam::<aws_account_id>:role/<role_name>

IBM Cloud Object Storage

When using IBM Cloud Object Storage (COS) as object storage, IAM Writer role is needed.

See the IBM Cloud Object Storage section on the storage page for a detailed setup guide.

Chunk Format

// Header
+-----------------------------------+
| Magic Number (uint32, 4 bytes)    |
+-----------------------------------+
| Version (1 byte)                  |
+-----------------------------------+
| Encoding (1 byte)                 |
+-----------------------------------+

// Blocks
+--------------------+----------------------------+
| block 1 (n bytes)  | checksum (uint32, 4 bytes) |
+--------------------+----------------------------+
| block 2 (n bytes)  | checksum (uint32, 4 bytes) |
+--------------------+----------------------------+
| ...                                             |
+--------------------+----------------------------+
| block N (n bytes)  | checksum (uint32, 4 bytes) |
+--------------------+----------------------------+

// Metas
+------------------------------------------------------------------------------------------------------------------------+
| #blocks (uvarint)                                                                                                      |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| #entries (uvarint) | minTs (uvarint) | maxTs (uvarint) | offset (uvarint) | len (uvarint) | uncompressedSize (uvarint) |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| #entries (uvarint) | minTs (uvarint) | maxTs (uvarint) | offset (uvarint) | len (uvarint) | uncompressedSize (uvarint) |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| ...                                                                                                                    |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| #entries (uvarint) | minTs (uvarint) | maxTs (uvarint) | offset (uvarint) | len (uvarint) | uncompressedSize (uvarint) |
+--------------------+-----------------+-----------------+------------------+---------------+----------------------------+
| checksum (uint32, 4 bytes)                                                                                             | 
+------------------------------------------------------------------------------------------------------------------------+

// Structured Metadata
+---------------------------------+
| #labels (uvarint)               |
+---------------+-----------------+
| len (uvarint) | value (n bytes) |
+---------------+-----------------+
| ...                             |
+---------------+-----------------+
| checksum (uint32, 4 bytes)      |
+---------------------------------+

// Footer
+-----------------------+--------------------------+
| len (uint64, 8 bytes) | offset (uint64, 8 bytes) |   // offset to Structured Metadata
+-----------------------+--------------------------+
| len (uint64, 8 bytes) | offset (uint64, 8 bytes) |   // offset to Metas
+-----------------------+--------------------------+