Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Configure

Versioning

Open source

About Grafana Mimir versioning

This topic describes our guarantees for this Grafana Mimir major release.

Flags, configuration, and minor version upgrades

Upgrading Grafana Mimir from one minor version to the next minor version should work, but we don’t want to bump the major version every time we remove a configuration parameter. We will keep deprecated features in place for two minor releases. You can use the deprecated_flags_inuse_total metric to generate an alert that helps you determine if you’re using a deprecated flag.

These guarantees don’t apply to experimental features.

Reading old data

The Grafana Mimir maintainers commit to ensuring that future versions can read data written by versions within the last two years. In practice, we expect to be able to read data written more than two years ago, but a minimum of two years is our guarantee.

API Compatibility

Grafana Mimir strives to be 100% compatible with the Prometheus HTTP API which is by default served by endpoints with the /prometheus HTTP path prefix /prometheus/*.

We consider any deviation from this 100% API compatibility to be a bug, except for the following scenarios:

Additional API endpoints for creating, removing, modifying alerts, and recording rules.
Additional APIs that push metrics (under /prometheus/api/push).
Additional API endpoints for management of Grafana Mimir, such as the ring. These APIs are not included in any compatibility guarantees.
Delete series API.

Experimental features

Grafana Mimir is an actively developed project and we encourage the introduction of new features and capabilities. Not everything in each release of Grafana Mimir is considered production-ready. We mark as “Experimental” all features and flags that we don’t consider production-ready.

We do not guarantee backwards compatibility for experimental features and flags. Experimental configuration and flags are subject to change.

The following features are currently experimental:

Compactor
- Enable cleanup of remaining files in the tenant bucket when there are no blocks remaining in the bucket index.
  - -compactor.no-blocks-file-cleanup-enabled
Ruler
- Tenant federation
- Disable alerting and recording rules evaluation on a per-tenant basis
  - -ruler.recording-rules-evaluation-enabled
  - -ruler.alerting-rules-evaluation-enabled
- Aligning of evaluation timestamp on interval (align_evaluation_time_on_interval)
Distributor
- Metrics relabeling
- OTLP ingestion path
- OTLP metadata storage
  - -distributor.enable-otlp-metadata-storage
- Using status code 529 instead of 429 upon rate limit exhaustion.
  - distributor.service-overload-status-code-on-rate-limit-enabled
- Set Retry-After header in recoverable error responses
  - -distributor.retry-after-header.enabled
  - -distributor.retry-after-header.base-seconds
  - -distributor.retry-after-header.max-backoff-exponent
Hash ring
- Disabling ring heartbeat timeouts
  - -distributor.ring.heartbeat-timeout=0
  - -ingester.ring.heartbeat-timeout=0
  - -ruler.ring.heartbeat-timeout=0
  - -alertmanager.sharding-ring.heartbeat-timeout=0
  - -compactor.ring.heartbeat-timeout=0
  - -store-gateway.sharding-ring.heartbeat-timeout=0
  - -overrides-exporter.ring.heartbeat-timeout=0
- Disabling ring heartbeats
  - -distributor.ring.heartbeat-period=0
  - -ingester.ring.heartbeat-period=0
  - -ruler.ring.heartbeat-period=0
  - -alertmanager.sharding-ring.heartbeat-period=0
  - -compactor.ring.heartbeat-period=0
  - -store-gateway.sharding-ring.heartbeat-period=0
  - -overrides-exporter.ring.heartbeat-period=0
- Reusable ingester push worker
  - -distributor.reusable-ingester-push-workers
Ingester
- Add variance to chunks end time to spread writing across time (-blocks-storage.tsdb.head-chunks-end-time-variance)
- Snapshotting of in-memory TSDB data on disk when shutting down (-blocks-storage.tsdb.memory-snapshot-on-shutdown)
- Out-of-order samples ingestion (-ingester.out-of-order-time-window)
- Shipper labeling out-of-order blocks before upload to cloud storage (-ingester.out-of-order-blocks-external-label-enabled)
- Postings for matchers cache configuration:
  - -blocks-storage.tsdb.head-postings-for-matchers-cache-ttl
  - -blocks-storage.tsdb.head-postings-for-matchers-cache-size (deprecated)
  - -blocks-storage.tsdb.head-postings-for-matchers-cache-max-bytes
  - -blocks-storage.tsdb.head-postings-for-matchers-cache-force
  - -blocks-storage.tsdb.block-postings-for-matchers-cache-ttl
  - -blocks-storage.tsdb.block-postings-for-matchers-cache-size (deprecated)
  - -blocks-storage.tsdb.block-postings-for-matchers-cache-max-bytes
  - -blocks-storage.tsdb.block-postings-for-matchers-cache-force
- CPU/memory utilization based read request limiting:
  - -ingester.read-path-cpu-utilization-limit
  - -ingester.read-path-memory-utilization-limit"
- Early TSDB Head compaction to reduce in-memory series:
  - -blocks-storage.tsdb.early-head-compaction-min-in-memory-series
  - -blocks-storage.tsdb.early-head-compaction-min-estimated-series-reduction-percentage
- Spread minimizing token generation strategy:
  - ingester.ring.token-generation-strategy
  - ingester.ring.spread-minimizing-zones
  - ingester.ring.spread-minimizing-join-ring-in-order
- Allow ingester’s Push() to return gRPC errors only: -ingester.return-only-grpc-errors.
Ingester client
- Per-ingester circuit breaking based on requests timing out or hitting per-instance limits
  - -ingester.client.circuit-breaker.enabled
  - -ingester.client.circuit-breaker.failure-threshold
  - -ingester.client.circuit-breaker.failure-execution-threshold
  - -ingester.client.circuit-breaker.period
  - -ingester.client.circuit-breaker.cooldown-period
Querier
- Use of Redis cache backend (-blocks-storage.bucket-store.metadata-cache.backend=redis)
- Streaming chunks from ingester to querier (-querier.prefer-streaming-chunks-from-ingesters)
- Streaming chunks from store-gateway to querier (-querier.prefer-streaming-chunks-from-store-gateways, -querier.streaming-chunks-per-store-gateway-buffer-size)
- Ingester query request minimisation (-querier.minimize-ingester-requests)
- Limiting queries based on the estimated number of chunks that will be used (-querier.max-estimated-fetched-chunks-per-query-multiplier)
- Max concurrency for tenant federated queries (-tenant-federation.max-concurrent)
Query-frontend
- -query-frontend.querier-forget-delay
- Instant query splitting (-query-frontend.split-instant-queries-by-interval)
- Lower TTL for cache entries overlapping the out-of-order samples ingestion window (re-using -ingester.out-of-order-allowance from ingesters)
- Use of Redis cache backend (-query-frontend.results-cache.backend=redis)
- Query blocking on a per-tenant basis (configured with the limit blocked_queries)
- Wait for the query-frontend to complete startup if a query request is received while it is starting up (-query-frontend.not-running-timeout)
Query-scheduler
- -query-scheduler.querier-forget-delay
Store-gateway
- Use of Redis cache backend (-blocks-storage.bucket-store.chunks-cache.backend=redis, -blocks-storage.bucket-store.index-cache.backend=redis, -blocks-storage.bucket-store.metadata-cache.backend=redis)
- -blocks-storage.bucket-store.series-selection-strategy
- Eagerly loading some blocks on startup even when lazy loading is enabled -blocks-storage.bucket-store.index-header.eager-loading-startup-enabled
Read-write deployment mode
API endpoints:
- /api/v1/user_limits
- /api/v1/cardinality/active_series
Metric separation by an additionally configured group label
- -validation.separate-metrics-group-label
- -max-separate-metrics-groups-per-user
Fetching TLS secrets from Vault for various clients (-vault.enabled)
Logger
- Rate limited logger support
  - log.rate-limit-enabled
  - log.rate-limit-logs-per-second
  - log.rate-limit-logs-burst-size
Memcached client
- Customise write and read buffer size
  - -<prefix>.memcached.write-buffer-size-bytes
  - -<prefix>.memcached.read-buffer-size-bytes
Timeseries Unmarshal caching optimization in distributor (-timeseries-unmarshal-caching-optimization-enabled)
Reusing buffers for marshalling write requests in distributors (-distributor.write-requests-buffer-pooling-enabled)
Using a worker pool for handling GRPC requests (-server.grpc.num-workers)
Limiting inflight requests to Distributor and Ingester via gRPC limiter:
- -distributor.limit-inflight-requests-using-grpc-method-limiter
- -ingester.limit-inflight-requests-using-grpc-method-limiter
Logging of requests that did not send any HTTP request: -server.http-log-closed-connections-without-response-enabled.

Deprecated features

Deprecated features are usable up until the release that indicates their removal. For details about what deprecated means, see Parameter lifecycle.

The following features or configuration parameters are currently deprecated and will be removed in Mimir 2.11:

Store-gateway
- -blocks-storage.bucket-store.chunk-pool-min-bucket-size-bytes
- -blocks-storage.bucket-store.chunk-pool-max-bucket-size-bytes
- -blocks-storage.bucket-store.max-chunk-pool-bytes
Querier, ruler, store-gateway
- -blocks-storage.bucket-store.bucket-index.enabled
Querier
- -querier.iterators and -querier.batch-iterators (Mimir 2.11 onwards will always use -querier.batch-iterators=true)

The following features or configuration parameters are currently deprecated and will be removed in Mimir 2.13: