Slide 13 of 15

Kafka integration

Kafka integration

The Kafka integration provides streaming platform monitoring for Apache Kafka clusters, including brokers, topics, and consumer groups. Kafka has many moving parts, but consumer lag is the metric that matters most, since growing lag means consumers are falling behind producers.

What it’s forMonitoring broker health, consumer lag, and message throughput
Under the hoodCollects broker and JVM metrics via a JMX exporter, plus consumer lag from a Kafka exporter

Metrics collected

The integration collects metrics across brokers, topics, and consumers:

  • Broker: JVM, requests, replication
  • Topics: Messages in/out, partitions
  • Consumer: Lag, offset, group status
  • ZooKeeper: Connections, latency (if used)

What to know

This integration offers:

  • Prebuilt dashboards: overview, topics, Kafka Connect
  • Prebuilt alerts: consumer lag, broker health
  • Partition distribution visibility
  • Watch cardinality, since thousands of topics and consumer groups generate many time series.

Set it up

Open this learning path in your Grafana Cloud stack for a fully interactive experience, or read through it to understand the process first.

Learning path

Monitor Kafka with Grafana Cloud

Welcome to the Kafka monitoring learning path that shows you how to use Grafana Alloy to send Kafka metrics to Grafana Cloud for comprehensive event streaming observability.

22 min
Beginner
Docs & blog posts

Open in Grafana Cloud

Complete this learning path directly in your Grafana Cloud stack with an interactive learning experience.

Open in Grafana Cloud
View on website

Path completed

Script

Kafka is complex, with brokers, topics, partitions, and consumer groups working together, and the integration gives you visibility into all of it.

Broker metrics are your cluster health foundation, covering JVM memory and garbage collection, request rates, and replication, since Kafka’s Java base means JVM health drives performance. But consumer lag is the metric that matters most. Growing lag means consumers are falling behind producers, which eventually affects your applications.

Setup requires JMX access to your brokers. Watch cardinality too, because environments with thousands of topics and groups generate huge numbers of time series, so filter to what matters.

This integration is essential for production Kafka, where catching problems early keeps streaming reliable.

Previous Next