Documentation Grafana Loki Release notes v3.2

v3.2

Grafana Labs and the Loki team are excited to announce the release of Loki 3.2. Explore Logs is also now Generally Available. Upgrade to Loki/GEL 3.2 to get the best possible experience with Explore Logs. Here’s a summary of new enhancements and important fixes.

For a full list of all changes and fixes, refer to the CHANGELOG.

Features and enhancements

Key features in Loki 3.2.0 include the following:

• *API: The Loki APIs have been updated to fail log queries when executed on instant query endpoint (#13421) (ce71f1c), add structured metadata to /detected_fields API (#13604) (ce02cc2), and avoid looking up unnecessary TSDB symbols during Volume API (#13960) (7c1a849).

• Query acceleration with Bloom filters (experimental): The Bloom filters feature introduced in Loki 3.0 remains experimental. Note that the Bloom Compactor component has been removed (#13969) (b75eacc) and replaced by the Bloom Planner and Bloom Builder components (#13997) (be9eb50).

• distributor: Ignore empty streams in distributor if all entries fail validation (#13674) (6c4b062), and limit to block ingestion until configured date (#13958) (b5ac6a0).

• Explore Logs Is now Generally Available (GA). For the best experience, you should be on Grafana 11.2 or later and Loki 3.2. This release includes enhancements to add _extracted suffix to detected fields conflicts (#13993) (ab1caea), collect and serve pre-aggregated bytes and counts (#13020) (467eb1b), and remove cardinality filter (#13652) (4f534d7).

• Helm: This release includes updates to the Helm charts to make gateway container port configurable. (#13294) (05176e4) and to support alibabacloud oss in the Helm chart (#13441) (3ebab6f). It also includes a breaking change to the Helm chart to support distributed mode and 3.0 (#12067).

• ingester: Ingester Stream Limit Improvements: Ingester stream limits now take into account “owned streams” and periodically update when the Ingester ring is changed. Non-owned streams are now also flushed when this update takes place. The stream limit calculation has also been updated for improved accuracy in multi-zone ingester deployments. (#13532) (ec34aaa).

• lambda-promtail: Add S3 log parser support for AWS GuardDuty (#13148) (2d92fff), build lambda with zip file (#13787) (9bf08f7), and ensure messages to Kinesis are usable by refactoring parsing of KinesisEvent to match parsing of CWEvents + code cleanup (#13098) (dbfb19b).

• loki: Add ability to disable AWS S3 dualstack endpoints usage(#13785) (bb257f5), not enforce max-query-bytes-read and max-querier-bytes-read in limited tripperware (#13406) (47f6ea5), and upgrade Prometheus (#13671) (b88583d).

• operator: Add alert for discarded samples (#13512) (5f2a02f), add support for the volume API (#13369) (d451e23), enable leader-election (#13760) (1ba4bff), and update Loki operand to v3.1.0 (#13422) (cf5f52d).

• storage: Convert WalSegmentWriter to io.ReadSeeker (#13340) (19c0509) and instrument failed chunk encoding/decoding (#13684) (5a87ccb).

• docs: Add getting started video for ingesting OTel logs (#13226) (5e560f9).

Other improvements include the following:

• chunks-inspect: Support structured metadata (#11506) (1834065).
• exporter: Include boolean values in limit exporter (#13466) (4220737).
• mempool: Replace sync.Mutex with sync.Once (#13293) (61a9854).
• metrics: Collect duplicate log line metrics (#13084) (40ee766).

Deprecations

One of the focuses of Loki 3.0 was cleaning up unused code and old features that had been previously deprecated but not removed. Loki 3.0 removed a number of previous deprecations and introduces some new deprecations. Some of the main areas with changes include:

• Deprecated storage options including the deprecation of the BoltDB store.

• Deprecated configuration options.

• API endpoint deprecations.

To learn more about breaking changes in this release, refer to the Upgrade guide.

Upgrade Considerations

For important upgrade guidance, refer to the Upgrade Guide and the separate Helm Upgrade Guide.

• BREAKING CHANGE - API: Fail log queries when executed on instant query endpoint (#13421).
• BREAKING CHANGE - blooms: Remove bloom compactor component (#13969).
• BREAKING CHANGE - docker: Remove wget from Promtail docker image(#15101).
• BREAKING CHANGE - Helm: Update Helm chart to support distributed mode and 3.0 (#12067).
• BREAKING CHANGE - Helm: Fix how we set imagePullSecrets for enterprise-gateway and admin-api. (#13761) (3be5a45).
• BREAKING CHANGE - jsonnet: Convert read statefulset into deployment for loki-simple-scalable (#13977).

Bug fixes

3.2.2 (2024-12-04)

• BREAKING CHANGE - docker: Remove wget from Promtail docker image(#15101).
• docker: Move from base-nossl to static. This PR removes the inclusion of glibc into most of the Docker images created by the Loki build system. (#15203).
• logql: Updated JSONExpressionParser not to unescape extracted values if it is JSON object. (#14499).
• promtail: Switch Promtail base image from Debian to Ubuntu to fix critical security issues (#15195).
• storage: Have GetObject check for canceled context. S3ObjectClient.GetObject incorrectly returned nil, 0, nil when the provided context is already canceled (#14420).

3.2.1 (2024-10-17)

• config: Copy Alibaba and IBM object storage configuration from common (#14315) (32a9bc0).
• logql: updated JSONExpressionParser not to unescape extracted values if it is JSON object. (backport release-3.2.x) (#14502) (e9bbaf3).
• promtail: Revert build image to Debian Bullseye to fix libc version issue in Promtail (#14386) (1e913df).
• storage: have GetObject check for canceled context (backport release-3.2.x) (#14422) (1b7dd95).

3.2.0 (2024-09-19)

• blooms: Cleanup temp blockdir in bloom compactor (#13622) (64215e1).
• blooms: Delete outdated metas during planning (#13363) (11e1976).
• blooms: Ensure tokenizer cache is reset between series (#13370) (04bc3a4).
• blooms: Fix eviction of multiple blockcache items (#13573) (c9950e3).
• blooms: Fix panic in bloom gateway (#13303) (66f97b2).
• blooms: Ignores bloom filtering errors in bounded shard query planning (#13285) (ede6941).
• blooms: Improve error wrap to make ignoreNotFound work when fetching blocks (#13656) (bd20171).
• blooms: Improves mempool metrics (#13283) (d36e1d5).
• blooms: Minor fixes and improvements for testing in dev (#13341) (d0f56ee).
• blooms: Remove backoff from notify planner (#13506) (e506995).
• blooms: Remove unused arg (#13343) (fcb9b28).
• blooms: Ship chunkrefs in task payload (#13677) (450bbce).
• blooms: Add logging to empty bloom (#13502) (c263a68).
• blooms: Skip empty blooms on reads (#13500) (bfa6955).
• blooms: Suppress error from resolving server addresses for blocks (#13385) (3ac2317).
• blooms: Use correct key to populate blockscache at startup (#13624) (2624a4b).
• blooms: Fix log line for fingerprint not found (#13555) (aeb23bb).
• blooms: Fix panic in BloomStore initialization (#13457) (5f4b8fc).
• blooms: Flaky test blockPlansForGaps (#13743) (37e33d4).
• blooms: Keep blocks referenced by newer metas (#13614) (784e7d5).
• blooms: Lint issues after merge to main (#13326) (7e19cc7).
• blooms: Use elements match in gapsBetweenTSDBsAndMetas test (#13722) (4cbe2a6).
• ci: Add cleanup step into job dist (#13801) (217f928).
• ci: Fixed release-please manifest (#13810) (f253db5).
• deps: Update github.com/axiomhq/hyperloglog digest to af9851f (#13806) (67295e0).
• deps: Update github.com/c2h5oh/datasize digest to aa82cc1 (#13807) (a93f38c).
• deps: Update github.com/docker/go-plugins-helpers digest to 45e2431 (#13808) (e5a3994).
• deps: Update github.com/grafana/jsonparser digest to ea80629 (#13814) (d5718eb).
• deps: Update module github.com/aliyun/aliyun-oss-go-sdk to v2.2.10+incompatible (#13861) (6f79194).
• deps: Update module github.com/azure/go-autorest/autorest/adal to v0.9.24 (#13862) (8041bd2).
• deps: Update module github.com/azure/go-autorest/autorest/azure/auth to v0.5.13 (#13863) (71c4421).
• deps: Update module github.com/baidubce/bce-sdk-go to v0.9.186 (#13864) (3c0e3e2).
• deps: Update module github.com/baidubce/bce-sdk-go to v0.9.187 (#13933) (56af84d).
• deps: Update module github.com/baidubce/bce-sdk-go to v0.9.188 (#14000) (79039a2).
• deps: Update module github.com/cespare/xxhash/v2 to v2.3.0 (main) (#13615) (cfc7b34).
• deps: Update module github.com/datadog/sketches-go to v1.4.6 (#13865) (1f3c467).
• deps: Update module github.com/docker/docker to v25.0.5+incompatible [security] (main) (#12279) (960c034).
• deps: Update module github.com/docker/docker to v27.1.1+incompatible [security] (main) (#13762) (f8bf3bb).
• deps: Update module github.com/docker/docker to v27.1.2+incompatible (#13872) (8ab4c20).
• deps: Update module github.com/efficientgo/core to v1.0.0-rc.3 (#14001) (90f7e5f).
• deps: Update module github.com/felixge/fgprof to v0.9.4 (#13870) (c68848f).
• deps: Update module github.com/fsouza/fake-gcs-server to v1.47.7 (#13935) (d43b2de).
• deps: Update module github.com/gogo/googleapis to v1.4.1 (#13871) (6da7eb5).
• deps: Update module github.com/gorilla/mux to v1.8.1 (main) (#13618) (19b288e).
• deps: Update module github.com/gorilla/websocket to v1.5.3 (#13873) (1eb8342).
• deps: Update module github.com/hashicorp/consul/api to v1.29.4 (#14002) (e11b244).
• deps: Update module github.com/ibm/go-sdk-core/v5 to v5.17.4 (#13892) (b6991f2).
• deps: Update module github.com/ibm/ibm-cos-sdk-go to v1.11.0 (#13893) (9b7e7e9).
• deps: Update module github.com/klauspost/pgzip to v1.2.6 (#13874) (fdea7a1).
• deps: Update module github.com/mattn/go-ieproxy to v0.0.12 (#13876) (775bf8e).
• deps: Update module github.com/ncw/swift to v2 (#13951) (246a1df).
• deps: Update module github.com/oschwald/geoip2-golang to v1.11.0 (#13934) (3bebba5).
• deps: Update module github.com/schollz/progressbar/v3 to v3.14.6 (#13884) (fb9cae4).
• deps: Update module github.com/tonistiigi/fifo to v1 (#13952) (96b5c79).
• deps: Update module github.com/workiva/go-datastructures to v1.1.5 (#13885) (d817aee).
• deps: Update module golang.org/x/text to v0.17.0 (main) (#13794) (df61482).
• deps: Update module golang.org/x/time to v0.6.0 (#13910) (dff00bd).
• detected fields: Detected fields incorrect type bug (#13515) (f6a94d3).
• detected fields: Remove query size limit for detected fields (#13423) (1fa5127).
• detected labels: Response when store label values are empty (#13970) (6f99af6).
• detected_labels: Add matchers to get labels from store" (#14012) (25234e8).
• detected_labels: Remove limit middleware for detected_labels (#13643) (2642718).
• docs: Fixed typo in ruler URL (#13692) (1476498).
• docs: Remove trailing backtick in verify-config for Loki 3.0 (#13640) (498f29a).
• Helm: Fix HPA ingester typo (#13158) (4ca9785).
• Helm: Fix extraObjects (#13107) (b7fcf2b).
• Helm: Fix imagePullSecrets for statefulset-results-cache (#13051) (8434b2f).
• Helm: Fixed memcached and provisioner templates (#13788) (1bf9791).
• Helm: Removed helm test (#13651) (ef03476).
• Helm: Update yaml file ./production/helm/loki/Chart.yaml (+1 other) (#13392) (b5b861c).
• Helm: Update yaml file ./production/helm/loki/values.yaml (+1 other) (#13426) (fc3904e).
• Helm: Querier address in SingleBinary mode (#13297) (29f1ea9).
• Helm: Update Loki v3 Helm statefulset-ingester.yaml template (#13118) (5b4e576).
• index gateways: Do not retain span logger created with index set initialized at query time (#14027) (bd25ac2).
• index gateway: Fix nil pointer dereference panic when using ruler in ring mode (#13436) (304db10).
• ingester: Fix panic in ingester.go (#13557) (dbff69a).
• ingester: Redo ingester profile tagging (#13239) (32097c8).
• ingester: Remove tenant label tagging from profiles to reduce cardinality (#13270) (f897758).
• ingester: Stream ownership check (#13314) (5ae5b31).
• ingester: Support multi-zone ingesters when converting global to local limits for streams in limiter.go (#13321) (e28c15f).
• ingester: Update fixed limit once streams ownership re-checked (#13231) (7ac19f0).
• LOgQL: AST left circular reference result in out of memory (#13501) (6dd6b65).
• LogQL: Improve execution speed for queries with label filters (#13922) (40f4f14).
• LogQL: Panic when parsing and extracting JSON key values (#13790) (5ef83a7).
• LogQL: Propagate headers/warnings/stats from quantile downstreams (#13881) (a0c7598).
• LogQL: Record datasample queries are limited query type (#13930) (ae938d0).
• LogQL: Return empty vector instead of nil for empty evaluator. (#13485) (08615bf).
• LogQL: Special case the return values from a sharded first/last_over_time query (#13578) (29a37d5).
• log results cache: Include pipeline wrapper disabled in cache key (#13328) (221491c).
• loki: Init internal server log along with loki’s server instance (#13221) (66b8c9b).
• loki-mixin: Fix latency panel for Index Gateway (#13629) (f586c00).
• loki-mixin: Various latency panels in operational dashboard should have ms unit type instead of seconds (#13260) (f5a9905).
• loki-mixin: Attribute OTLP route correctly to write path (#13943) (b91b782).
• loki-mixins: Incorrect pod matcher for compactor in mixin when using ssd mode (#12846) (515e13c).
• operator: Allow structured metadata only if V13 schema provided (#13463) (3ac130b).
• operator: Don’t overwrite annotations for LokiStack ingress resources (#13708) (f523530).
• operator: Remove duplicate conditions from status (#13497) (527510d).
• operator: Set object storage for delete requests when using retention (#13562) (46de4c1).
• operator: Skip updating annotations for serviceaccounts (#13450) (1b9b111).
• operator: Support v3.1.0 in OpenShift dashboards (#13430) (8279d59).
• operator: Watch for CredentialsRequests on CCOAuthEnv only (#13299) (7fc926e).
• querier: Add a retry middleware to all the stats handlers (#13584) (7232795).
• querier: Adjust tailer loop criteria so it is actually re-tested (#13906) (dabbfd8).
• querier: Fix retry code to handle grpc status codes. updated newer stats retries to be wrapped with spans (#13592) (d3e1edb).
• querier: Fixes span name of serializeRounTripper (#13541) (4451d56).
• querier: Remove retries on the stats handlers because they already retry (#13608) (1008315).
• query engine: Include lines with ts equal to end timestamp of the query range when executing range aggregations (#13448) (e0ca67d).
• retry: Fix retries when using protobuf encoding (#13316) (a457c5d).
• ruler: Protect ruler remote-write overrides map with a mutex when creating new appenders (#13676) (e9a9c60).
• sharding: Use without() grouping when merging avg_over_time shard results (#12176) (eb8a363).
• storage: Handle block offset exceeding chunk length in memchunk.go (#13661) (d42476a).
• storage: Read “404” as object not exist (#13901) (3c9c647).
• storage: Separates directory creation from permission checks (#13248) (1086783).
• storage: Try reading chunks which have incorrect offset for blocks (#13720) (7e224d5).
• structured metadata: Sanitize structured metadata at query time (#13983) (3bf7fa9).
• WAL: Fix a bug where AppendRequest with no entries triggers flush (#13672) (8a3ae22).
• WAL: Properly reset wal segment writer (#13468) (6ea83b4).