Menu
Grafana Cloud

How policies are evaluated

When a trace is received, it is evaluated against all your policies at once to make a single, final decision: keep (sample) or drop. The logic follows two core principles.

Principle 1: Keep if it matches any keep policy (the “or” logic)

Think of your keep policies as a giant OR condition. A trace only needs to match one of these policies to be flagged for keeping. It does not need to match all of them.

For example, if you have these policies:

  • Keep traces with errors.
  • Keep traces with latency > 2 seconds.
  • Keep 10% of all remaining traces.

A trace with an error is kept.

A trace with a 3-second latency is kept. It does not matter if the slow trace also had an error; one match is enough.

The 10% probabilistic policy acts as a catch-all for traces that do not match any of your more specific rules.

Principle 2: Drop always wins

A drop policy is an absolute veto. If a trace matches any drop policy, it is discarded immediately, even if it also matches one or more keep policies.

For example, imagine you add this policy:

  • Drop all traces from the /grafana endpoint.

If a slow, erroneous trace comes from the /grafana endpoint, it is dropped. The drop rule takes priority over all keep rules.

Example

Let’s look at how the system evaluates a few different traces with the policies from above.

Trace DetailsEvaluationFinal Decision
Trace A: duration: 3s, endpoint: /api/checkoutMatches the latency > 2s keep policy.KEEP
Trace B: duration: 150ms, endpoint: /api/productsDoesn’t match any specific keep or drop policy. It’s now subject to the 10% probabilistic rule.KEEP (if it’s in the 10%) or DROP (if it’s in the other 90%)
Trace C: duration: 3s, endpoint: /grafanaMatches both the latency > 2s (keep) policy AND the /grafana (drop) policy.DROP (The drop policy wins)

In summary, just remember these rules.

  • A trace is evaluated against all policies at once. A trace is kept if it matches at least one keep policy.
  • A trace is dropped if it matches any drop policy.
  • If a trace matches both a keep and a drop policy, drop always wins.

Policy example using code

The following code snippet is example of how the policies above can be defined using JSON.

# Policy 1: Keep traces with latency > 2 seconds

{
  "threshold_ms": 2000,
}


# Policy 2: Keep 10% of all traces as a general sample

{
  "sampling_percentage": 10
}

# Policy 3: Drop all traces from the /grafana endpoint

{
    "drop_sub_policy": [
    {
      "name": "healthcheck-drop",
      "type": "string_attribute",
      "string_attribute": {
        "key": "http.target",
        "values": [
          "/grafana"
        ]
      }
    }
  ]
}

# Policy 4: Keep all traces that have an error

{
      "name": "error-keep",
      "type": "status_code",
 "status_codes": [
    "ERROR"
  ]
}

How to interpret this code

  1. List of Policies: The system evaluates a trace against this entire list.
  2. Drop: The healthcheck-drop policy is explicitly configured to action: drop. When a trace with http.target: /grafana is processed, this rule provides the definitive drop decision that overrides any keep decisions from other policies.
  3. Keep: For a trace that is not dropped, the system only needs one of the other policies. For example, high-latency-keep, error-keep, or probabilistic-catch-all to return a keep decision.