This is documentation for the next version of Grafana Mimir documentation. For the latest stable release, go to the latest version.

Documentation Grafana Mimir Set up Deploy with Jsonnet and Tanka Configure ingest storage architecture

Configure ingest storage architecture

The ingest storage is the next generation architecture of Grafana Mimir. It decouples the Mimir read and write paths using Apache Kafka or a Kafka-compatible backend.

To enable ingest storage, set the following Jsonnet:

jsonnet Copy

{
  _config+:: {
    ingest_storage_enabled: true,
  }
}

Kafka connection options

The Jsonnet configures some Kafka connection parameters by default:

• Kafka topic: ingest
• Auto-created partitions: 1000 (when the topic doesn’t exist)
• Kafka addresses: kafka.<namespace>.svc.<cluster_domain>:9092

To customize the Kafka connection configuration, override the default settings:

jsonnet Copy

{
  _config+:: {
    ingest_storage_enabled: true,
  },

  // Override producer and consumer addresses for external Kafka
  ingest_storage_kafka_producer_address:: 'kafka-broker-1:9092,kafka-broker-2:9092',
  ingest_storage_kafka_consumer_address:: 'kafka-broker-1:9092,kafka-broker-2:9092',

  // Override Kafka client configuration
  ingest_storage_kafka_client_args+:: {
    'ingest-storage.kafka.topic': 'mimir-ingest',
    'ingest-storage.kafka.auto-create-topic-default-partitions': 500,
  },
}

Kafka partition sizing

The configured topic must have at least as many partitions as the number of ingesters in one zone.

Refer to Configure the Grafana Mimir Kafka backend for more details about Kafka configurations.