This is documentation for the next version of Mimir. For the latest stable release, go to the latest version.

Documentation Grafana Mimir Configure Kafka

Configure the Grafana Mimir Kafka backend

Grafana Mimir supports using Kafka for the first layer of ingestion. This is an experimental feature released in Mimir 2.14. This page is incomplete. It will be updated as the ingest storage feature matures and moves out of the experimental phase.

Different Kafka backend implementations

Some Kafka-compatible implementations have different behavior for the Kafka API. To set up Mimir to work with different Kafka backends, you need to configure some parameters. Here are the Kafka flavors and additional configurations needed to set them up in Mimir.

Apache Kafka

Use the default options with Apache Kafka. No additional configuration is needed.

Confluent Kafka

Use the default options with Confluent Kafka. No additional configuration is needed.

Warpstream

Configure the following CLI flags or their YAML equivalent.

Copy

-ingest-storage.kafka.use-compressed-bytes-as-fetch-max-bytes=false