limit stage is a rate-limiting stage that throttles logs based on several options.
Limit stage schema
This pipeline stage places limits on the rate or burst quantity of log lines that Promtail pushes to Loki.
The concept of having distinct burst and rate limits mirrors the approach to limits that can be set for Loki’s distributor component:
ingestion_burst_size_mb, as defined in limits_config.
limit: # The rate limit in lines per second that Promtail will push to Loki [rate: <int>] # The cap in the quantity of burst lines that Promtail will push to Loki [burst: <int>] # When drop is true, log lines that exceed the current rate limit will be discarded. # When drop is false, log lines that exceed the current rate limit will only wait # to enter the back pressure mode. [drop: <bool> | default = false]
The following are examples showing the use of the
limit stage configurations.
Match a line and throttle
Given the pipeline:
- limit: rate: 10 burst: 10
Would throttle any log line.
Match a line and drop
Given the pipeline:
- limit: rate: 10 burst: 10 drop: true
Would throttle any log line and drop logs when rate limit.
Related Enterprise Logs resources
Grafana Enterprise Logs: Logging with security and scale
Join us for this webinar, which will cover: Challenges with logging as organizations scale and the volume of logs explodes, how Grafana Enterprise Logs enables organizations to make logs available to any team members who need them, features available in GEL and how to get access, a live product demo so you can see GEL for the first time
VIDEO: Watch this first-look demo of the new Grafana Enterprise Logs
Based on Loki, Grafana Enterprise Logs is part of the Grafana Enterprise Stack for composing and scaling observability on your own infrastructure.
Introducing Grafana Enterprise Logs, a core part of the Grafana Enterprise Stack integrated observability solution
Powered by the Loki open source project, the Enterprise Logs offering joins metrics and dashboards in our enterprise-ready stack for self-managed observability.