Open source at Grafana Labs: 2024 year in review

Open source has always been the bedrock for everything we build here at Grafana Labs, going all the way back to Grafana creator Torkel Ödegaard’s first commit in December 2013.

Ten years after Grafana Labs was founded, open source continued to be our driving force as we worked to develop and evolve our core OSS tools and technologies in 2024. Combine that with our contributions to OpenTelemetry and Prometheus, as well as our new open source projects like Grafana Alloy, and it all adds up to a whole lot of OSS goodness that aligns with our “big tent” philosophy.

“We’re building bridges between different observability approaches, contributing upstream to multiple open source communities, and ensuring teams can use the right tool for the right job,” Grafana Labs CTO Tom Wilkie recently said. “This is how open source should work – inclusive, interoperable, and focused on solving real user needs.”

Let’s take a look at some of the biggest highlights from 2024.

Grafana LGTM Stack updates

Loki

With the Grafana Loki 3.0 release, our log aggregation system hit a major milestone in 2024. Six years in and more than 24,000 GitHub stars later, we continued to push the limits of what you can do with Loki.

• Accelerated query results: The 3.0 release introduced Bloom filters, an experimental feature intended to help return “needle in a haystack” queries faster. After working with early adopters and seeing how Bloom filters worked at scale, we decided to adjust our approach in Loki 3.3 to leverage structured metadata, making it faster to build, download, and query.
• Native OpenTelemetry support. Say good-bye to the Loki Exporter! Now that Loki has native OTel support, you get a simpler ingestion pipeline by removing the Loki Exporter from the process. You also get a better querying experience since you can interact with all the OpenTelemetry attributes and log event metadata at query time without having to do any deserialization.

Grafana

Even with more than a decade of development and yet another major release under our belt, we still love to find new ways to improve Grafana for collecting, correlating, and visualizing data.

Between the announcement of Grafana 11.0 at GrafanaCON 2024 in April and the 11.4 release just this month, there were scads of upgrades big and small. Here are just a few of the noteworthy ones.

• Custom visualizations. Canvas panels have continued to grow in popularity since they were introduced in Grafana 9.x, and this year we made a number of improvements to them, including high-demand flowcharting features.
• Easier management at scale. With the addition of subfolders, user-based authentication with Azure Monitor, and an improved dashboard-to-PDF feature for large dashboards, we’re making it easier for large organizations to manage their environment and safely share insights across their orgs.
• Scenes-powered dashboards. The architecture for Grafana dashboards has migrated to the Scenes library, giving you more stable, dynamic, and flexible dashboards.
• Text wrapping. You asked for it, and we were happy to ship it. We’ve enabled text wrapping within cells for the table panel, making convoluted, long lines of text easier to read.
• Grafana Alerting upgrades. There has been a big focus on helping you simplify how you manage alerts in Grafana 11.x, from Keep last state to a holistic view of your alerts to alert templates. We also added role-based access control for notifications and Grafana-managed recording rules.

Tempo

Grafana Tempo, our easy-to-use, and high-scale distributed tracing backend, had three minor releases in 2024, ushering in lots of performance and functionality improvements along the way.

• TraceQL metrics. An experimental feature introduced in Tempo 2.4, TraceQL metrics creates metrics from traces, much in the same way LogQL creates metrics from logs. Initially supporting the rate() function, Tempo 2.5 added quantile_over_time() and histogram_over_time() so you can aggregate numerical values as well.
• More TraceQL features. Building on the vParquet4 backend, we’ve added support for querying events, span links, and arrays.
• Better performance. The Tempo 2.6 release included several performance improvements for TraceQL, and it also came with a notable reduction of memory usage due to polling improvements.
• gRPC streaming. In Tempo 2.5 support for gRPC streaming was added to most Tempo HTTP endpoints including TraceQL search, tag name lookups, tag value lookups and TraceQL metrics.

Mimir

The four Grafana Mimir 2.1x minor releases represented another step forward for our Prometheus-compatible time series database for long-term storage:

• OTLP endpoint support. By implementing native translation, we’ve improved OTLP ingestion performance significantly – cutting distributor memory usage by 30% and CPU usage by 8%.
• PromQL engine. This experimental feature, currently a release candidate, decreases the peak memory consumption of Mimir’s querier component, so that queries over millions of time series no longer trigger out-of-memory errors or require users to massively over-resource their queriers.
• Kafka-based ingest storage. Kafka, the open source distributed event streaming system, is a critical tool for many large organizations. This experimental feature can help manage long-term costs associated with storing event streams.
• Query scheduler routing optimization. This update improves overall query execution latency, helping you get the answers you need faster.

Learn more

You can check out our documentation for details on all of the updates to the Grafana LGTM Stack and more, including valuable upgrades we’ve made to our other OSS projects such as Grafana k6 for performance testing and Grafana Pyroscope for continuous profiling.

Data analysis without queries: Explore apps

The Explore apps suite, announced at ObservabilityCon 2024, provides an opinionated, intuitive, and queryless user experience for quickly answering questions about your metrics, logs, traces, and profiles through simple, point-and-click interactions. The suite of apps is designed for all users, whether you’re unfamiliar with querying languages or an expert just looking to simplify the process of finding the data you need.

Available in Grafana OSS, Grafana Enterprise, and Grafana Cloud, the Explore apps include Explore Metrics, Explore Logs, Explore Traces, and Explore Profiles.

Check out a demo of how Explore apps work.

Introducing Grafana Alloy

The latest open source project from Grafana Labs, Grafana Alloy is our distribution of the OpenTelemetry Collector with built-in Prometheus pipelines and support for metrics, logs, traces, and profiles.

Announced at GrafanaCon 2024, Alloy uses the same components, code, and concepts that were first introduced in Grafana Agent Flow. Since then, we’ve rolled out new features, including live debugging, which supports real-time monitoring of data pipelines and more efficient troubleshooting.

Note: Grafana Agent, the predecessor to Alloy, is expected to reach end-of-life phase on Nov. 1, 2025. For more information, read our FAQ blog about migration from Grafana Agent to Alloy.*

OpenTelemetry

OpenTelemetry stands out as one of the fastest growing open source projects in the cloud native arena, with 43% increase in code commits in GitHub and a 100% increase in search volume on Google in 2024 according to a recent OpenTelemetry report.

Grafana Labs continues to be active in the OpenTelemetry community in an effort to help drive innovation and adoption. For example, in June, Grafana Labs engineers released new open source code that allows users to translate Datadog metric formats into native OTLP format. These metrics that are collected and translated by the OpenTelemetry Datadog receiver can be sent to any OpenTelemetry-compatible metrics system, whether it is Prometheus, Grafana Mimir, or another backend database. The receiver, marked as experimental, is available as both a Grafana Alloy and OpenTelemetry component.

We’ve also worked to ensure our open source technology and tools can work seamlessly with OpenTelemetry:

OpenTelemetry + metrics

In Grafana, Explore Metrics, which allows users to browse and analyze Prometheus-compatible metrics without writing PromQL queries, expanded to automatically handle OpenTelemetry metrics, eliminating the need for users to create separate queries for each system. This unified approach means users can access and visualize more types of metrics through a single, consistent interface regardless of whether the data is in Prometheus or OpenTelemetry format. This development represents Grafana Labs’ larger commitment to improving compatibility between OpenTelemetry and Prometheus while meeting users where they are no matter what formats they use.

OpenTelemetry + profiles

This year, the OpenTelemetry community took significant steps towards establishing profiling as a core OTel signal. To align with those efforts, we rolled out experimental support for OpenTelemetry profiles in the Grafana Pyroscope 1.10.0 release last month.

OpenTelemetry + eBPF

In November 2023, we announced the general availability of Grafana Beyla, our open source OpenTelemetry and Prometheus eBPF auto-instrumentation tool to help you easily get started with application observability. In the year since, we’ve been busy growing the community to help with important patches and other contributions, such as the Beyla Helm chart. In fact, there is a 10:1 ratio of external contributors to Grafanistas contributing to Beyla today.

Since our 1.0 release, which tracked the HTTP and gRPC protocols, we’ve added support for HTTP2, SQL, Redis, and Kafka. We also started tracking network and connection metrics, which allows users to build solutions for service graphs.

We worked hard to pick the right defaults, balancing ease of use with generating too many metrics. As a result of all this work, we saw the full OpenTelemetry Demo instrumented with a single Beyla daemonset deployment earlier this year. The one Beyla instance produced service-level application metrics for all the different technologies and protocols used to implement the services in the OpenTelemetry Demo.services, while keeping the bare-bones uninstrumented applications talking to each other.

Prometheus 3.0

OpenTelemetry and Prometheus are staples of today’s observability landscape. In fact, according to our 2024 Observability Survey, an overwhelming majority of industry practitioners are investing in Prometheus (89%) or OpenTelemetry (85%), and almost 40% are using both.

Grafana Labs is deeply committed to both projects — and we are committed to making both projects work together. To that end, Grafana Labs is the No. 1 company contributor to Prometheus and a top 10 contributor to OpenTelemetry. Several Grafanistas even double as Prometheus maintainers, helping to support the release of Prometheus 3.0.

That 3.0 release marked a big step forward in Prometheus supporting OpenTelemetry, and it also mirrors our efforts to make our latest projects, Beyla and Alloy, support both projects from the outset.

We look forward to working with the open source community even more in the new year!

We’d love to hear your thoughts on open source and the state of observability. Click the button below to take our annual Observability Survey today!