Grafana Mimir version 2.4 release notes
Grafana Labs is excited to announce version 2.4 of Grafana Mimir.
The highlights that follow include the top features, enhancements, and bugfixes in this release. For the complete list of changes, see the changelog.
Note: If you are upgrading from Grafana Mimir 2.3, review the list of important changes that follow.
Features and enhancements
Query-scheduler ring-based service discovery: The query-scheduler is an optional, stateless component that retains a queue of queries to execute, and distributes the workload to available queriers. The use the query-scheduler, query-frontends and queriers are required to discover the addresses of the query-scheduler instances.
In addition to DNS-based service discovery, Mimir 2.4 introduces the ring-based service discovery for the query-scheduler. When enabled, the query-schedulers join their own hash ring (similar to other Mimir components), and the query-frontends and queriers discover query-scheduler instances via the ring.
Ring-based service discovery makes it easier to set up the query-scheduler in environments where you can’t easily define a DNS entry that resolves to the running query-scheduler instances. For more information, refer to query-scheduler configuration.
New API endpoint exposes per-tenant limits: Mimir 2.4 introduces a new API endpoint, which is available on all Mimir components that load the runtime configuration. The endpoint exposes the limits of the authenticated tenant. You can use this new API endpoint when developing custom integrations with Mimir that require looking up the actual limits that are applied on a given tenant. For more information, refer to Get tenant limits.
New TLS configuration options: Mimir 2.4 introduces new options to configure the accepted TLS cipher suites, and the minimum versions for the HTTP and gRPC clients that are used between Mimir components, or by Mimir to communicate to external services such as Consul or etcd.
You can use these new configuration options to override the default TLS settings and meet your security policy requirements. For more information, refer to Securing Grafana Mimir communications with TLS.
Maximum range query length limit: Mimir 2.4 introduces the new configuration option
-query-frontend.max-total-query-lengthto limit the maximum range query length, which is computed as the query’s
starttimestamp. This limit is enforced in the query-frontend and defaults to
The new configuration option allows you to set different limits between the received query maximum length (
-query-frontend.max-total-query-length) and the maximum length of partial queries after splitting and sharding (
The following experimental features have been promoted to stable:
Helm chart improvements
mimir-distributed Helm chart is the best way to install Mimir on Kubernetes. As part of the Mimir 2.4 release, we’re also releasing version 3.2 of the
mimir-distributed Helm chart.
Notable enhancements follow. For the full list of changes, see the Helm chart changelog.
- Added support for topologySpreadContraints.
- Replaced the default anti-affinity rules with topologySpreadContraints for all components which puts less restrictions on where Kubernetes can run pods.
- Important: if you are not using the sizing plans (small.yaml, large.yaml, capped-small.yaml, capped-large.yaml) in production, you must reintroduce pod affinity rules for the ingester and store-gateway. This also fixes a missing label selector for the ingester.
Merge the following with your custom values file:yaml
ingester: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: target operator: In values: - ingester topologyKey: "kubernetes.io/hostname" - labelSelector: matchExpressions: - key: app.kubernetes.io/component operator: In values: - ingester topologyKey: "kubernetes.io/hostname" store_gateway: affinity: podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: target operator: In values: - store-gateway topologyKey: "kubernetes.io/hostname" - labelSelector: matchExpressions: - key: app.kubernetes.io/component operator: In values: - store-gateway topologyKey: "kubernetes.io/hostname"
- Updated the anti affinity rules in the sizing plans (small.yaml, large.yaml, capped-small.yaml, capped-large.yaml). The sizing plans now enforce that no two pods of the ingester, store-gateway, or alertmanager StatefulSets are scheduled on the same Node. Pods from different StatefulSets can share a Node.
- Support for Openshift Route resource for nginx has been added.
In Grafana Mimir 2.4, the default values of the following configuration options have changed:
-distributor.remote-timeouthas changed from
-distributor.forwarding.request-timeouthas changed from
-blocks-storage.tsdb.head-compaction-concurrencyhas changed from
- The hash-ring heartbeat period for distributors, ingesters, rulers, and compactors has increased from
In Grafana Mimir 2.4, the following deprecated configuration options have been removed:
- The YAML configuration option
- The CLI flag
-ingester.ring.join-afterand its respective YAML configuration option
- The CLI flag
-querier.shuffle-sharding-ingesters-lookback-periodand its respective YAML configuration option
With Grafana Mimir 2.4, the anonymous usage statistics tracking is enabled by default. Mimir maintainers use this anonymous information to learn more about how the open source community runs Mimir and what the Mimir team should focus on when working on the next features and documentation improvements. If possible, we ask you to keep the usage reporting feature enabled. In case you want to opt-out from anonymous usage statistics reporting, refer to Disable the anonymous usage statistics reporting.
- PR 2979: Fix remote write HTTP response status code returned by Mimir when failing to write only to one ingester (the quorum is still honored when running Mimir with the default replication factor of 3) and some series are not ingested because of validation errors or some limits being reached.
- PR 3005: Fix the querier to re-balance its workers connections when a query-frontend or query-scheduler instance is terminated.
- PR 2963: Fix the remote read endpoint to correctly support the
Accept-Encoding: snappyHTTP request header.