Blocking Queries

In certain situations, you may not be able to control the queries being sent to your Loki installation. These queries may be intentionally or unintentionally expensive to run, and they may affect the overall stability or cost of running your service.

You can block queries using per-tenant overrides, like so:

overrides:
  "tenant-id":
    blocked_queries:
      # block this query exactly
      - pattern: 'sum(rate({env="prod"}[1m]))'

      # block any query matching this regex pattern 
      - pattern: '.*prod.*'
        regex: true

      # block all metric queries
      - types: metric

      # block any filter or limited queries matching this regex pattern 
      - pattern: '.*prod.*'
        regex: true
        types: filter,limited

      # block any query that matches this query hash
      - hash: 2943214005          # hash of {stream="stdout",pod="loki-canary-9w49x"}
        types: filter,limited

NOTE: changes to these configurations do not require a restart; they are defined in the runtime configuration file.

The available query types are:

metric: a query with an aggregation, e.g. sum(rate({env="prod"}[1m]))
filter: a query with a log filter, e.g. {env="prod"} |= "error"
limited: a query without a filter or a metric aggregation

The hash option uses a 32-bit FNV-1 hash of the query string, represented as a 32-bit unsigned integer. This can often be easier to use than query strings that are long or require lots of string escaping. A query_hash field is logged with every query request in the query-frontend and querier logs, for easy reference. Here’s an example log line:

level=info ts=2023-03-30T09:08:15.2614555Z caller=metrics.go:152 component=frontend org_id=29 latency=fast 
query="{stream=\"stdout\",pod=\"loki-canary-9w49x\"}" query_hash=2943214005 query_type=limited range_type=range ...

Note: the order of patterns is preserved, so the first matching pattern will be used