The open source journey: Enabling real-time Kafka data visualization in Grafana

Have you ever wanted to visualize live Apache Kafka data directly in Grafana, eliminating the need for complex intermediate storage or external services?

In this session, Hamed Karbasi, a data engineer at 0+X, chronicles the open source development journey of the Grafana Kafka data source plugin, which allows users to query and visualize real-time Kafka data streams instantly.

See a live demonstration of how to get started with the plugin to integrate Kafka data for observability, including the steps you take to:

• Configure the plugin to ingest data from specific Kafka topics
• Successfully handle flexible data formats, specifically Protobuf, JSON, and AVRO messages
• Build dynamic, real-time dashboards for comprehensive streaming data visualization

By highlighting the strong community collaboration and support available for building new Grafana plugins that address real-world needs, Hamed hopes to inspire other developers to contribute their own integrations to the Grafana ecosystem.