Grafana Mimir 3.0 release: performance improvements, a new query engine, and more

In 2022, we introduced Grafana Mimir, our open source, horizontally scalable, multi-tenant time series database (TSDB) designed for long-term storage of Prometheus and OpenTelemetry metrics. Over the years, Mimir has become a go-to metrics backend within the open source community, with 30 project maintainers and more than 4.7k GitHub stars.

From the start, we were clear about our ambitions to make Mimir the most scalable and performant open source TSDB in the world, allowing users to scale to 1 billion active series and beyond. And today, we’re excited to share the latest milestone in that journey with the launch of Grafana Mimir 3.0.

Three years in the making, Mimir 3.0 marks a new era for the metrics backend, delivering dramatic improvements in reliability, performance, and cost efficiency.

Here, we’ll take a closer look at the latest enhancements and architectural updates in Mimir 3.0, which are also now available in Grafana Cloud Metrics, the fully managed, highly scalable metrics service powered by Mimir.

Mimir 3.0 improvements: faster performance, optimized resource usage

Here at Grafana Labs, we operate some of the largest Mimir clusters in the world. We also work closely with customers who operate their own Mimir environments at scale. With Mimir 3.0, we applied what we’ve learned from both of these experiences, and focused on three main areas for improvement:

• Reliability: By separating read and write operations, Mimir 3.0 eliminates cross-path dependencies, meaning your queries stay fast and stable even under heavy ingestion loads.
• Performance: The Mimir Query Engine — a new alternative to the traditional Prometheus query engine — streams query results instead of loading entire datasets into memory, resulting in faster execution and improved efficiency at scale.
• Cost optimization: Thanks to these performance optimizations, large clusters in our internal environment are experiencing up to 15% lower resource usage, while also improving performance and maintaining higher reliability.

A new, decoupled architecture

Under the hood, these performance improvements are driven by one of the most significant changes in Grafana Mimir 3.0: a decoupled architecture that further separates the read and write paths.

In previous versions of Mimir, the ingester sat in both the read and write paths, which meant heavy query loads could hinder ingestion performance. By introducing Apache Kafka, an open source distributed event streaming platform developed by the Apache Software Foundation, as an asynchronous buffer between ingestion and query, Mimir now allows each path to scale independently.

This architectural design delivers faster queries, smoother ingestion, and greater reliability for operators running massive clusters, enabling them to spend more time optimizing business outcomes and less time troubleshooting issues.

The role of ingest storage

Ingest storage — a foundational component of the new Mimir architecture — separates reads from writes so that one no longer affects the other. For example, a spike in query volume won’t slow down ingestion, or vice versa. This not only improves performance, but simplifies scaling and operational maintenance.

As shown in the graph below, the probability of a read path outage due to random ingester failures was significantly reduced, especially during the early stages of ingester failures, when we tested the new architecture internally.

To learn more about the ingest storage, and the redesigned architecture in Mimir 3.0, check out this blog post.

A more memory-efficient query engine 

Rolled out in Grafana Mimir 2.17, Mimir Query Engine is also a foundational feature in Mimir 3.0, offering faster, more memory-efficient performance for users.

Before MQE, Mimir exclusively used Prometheus’ PromQL engine to evaluate queries. While that standard PromQL engine generally worked well, it processes samples in bulk, which could lead to unpredictable memory usage. MQE changes all that, processing queries in a streaming fashion and loading only the necessary samples at each step. This reduces peak memory usage by up to 92%, resulting in faster queries and improved reliability, especially under heavy loads.

MQE is 100% PromQL-compatible and is the default query engine in Mimir 3.0. To learn more about MQE, check out this blog post and our documentation.

Upgrading to Mimir 3.0

Because this release includes significant architectural changes, we recommend planning your upgrade early. Before you begin, ensure you have the following:

• An existing Mimir cluster running with the classic architecture
• Access to deploy and configure a second Mimir cluster
• Write clients, such as Prometheus or the OpenTelemetry Collector, that you can configure to send data to multiple endpoints
• Read clients, such as Grafana, that you can reconfigure to point to a new endpoint
• The ability to modify Helm or Jsonnet configurations for both clusters

Please reference our upgrade guide and release notes for more details.

A big thank you to our community!

As always, we couldn’t have reached this milestone without our amazing community. Your feedback, bug reports, and operational insights continue to shape the evolution of Mimir.

We also want to give a special shout-out to Grafana Labs team members Armand Grillet, Zhehao Zhou, and Taylor Cole for their help with this post and with the release, as a whole.

Mimir 3.0 is built on everything we’ve learned together — and we can’t wait to see what you build with it. Thank you for helping us push the limits of open observability. Here’s to a faster and even more scalable Grafana Mimir!

Grafana Cloud is the easiest way to get started with metrics, logs, traces, dashboards, and more. We have a generous forever-free tier and plans for every use case. Sign up for free now!