Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Observe and actbreadcrumb arrow Agent Observabilitybreadcrumb arrow Guidesbreadcrumb arrow Guard requests
Grafana Cloud

Set up guards

Guards are synchronous checks that run on the request path before an LLM call. Use them to block harmful prompts, redact sensitive data, or filter dangerous tool calls — all before the model sees the request.

Guards reuse the same evaluator types as online evaluation (LLM judge, regex, JSON schema, heuristic) but execute synchronously and return an allow/deny decision to the SDK.

Before you begin

  • Agent Observability is enabled on your Grafana Cloud stack and receiving generation data.
  • If you plan to use LLM judge guards, choose a provider and model available in your stack. Refer to Choose a judge provider.
  • You have the Agent Observability Admin role, or a custom Grafana role with grafana-agento11y-app.guards:write and the read access needed for evaluator testing.

How guards work

  1. Your application SDK calls Agent Observability before each LLM invocation.
  2. Agent Observability evaluates the request against enabled guard rules in priority order.
  3. Each matching rule can redact the input with regex patterns, check tool calls against a block list, or run evaluators against the content.
  4. The response tells the SDK whether to proceed (allow) or abort (deny), and optionally includes a sanitized copy of the input.

If the guard service is unreachable, the SDK proceeds by default (fail-open). You can change this to fail-closed in your SDK configuration.

Create a guard

  1. Navigate to Evaluations > Guards in Agent Observability.
  2. Click New guard.
  3. Choose Create manually, Use a template, or Use Assistant.
  4. Enter a guard ID (for example, pii_redaction or block_dangerous_tools).
  5. Choose a guard type:

Evaluator guard

Runs one or more evaluators on the request content and denies or warns based on the result.

  1. Select Evaluator as the guard type.
  2. Attach one or more evaluators (existing evaluators from the Evaluators tab).
  3. Set Action on fail to deny (blocks the request) or warn (logs but allows).

Redact guard

Applies regex patterns to redact content before the request reaches the model.

  1. Select Redact as the guard type.
  2. Add one or more patterns:
    • Regex — the pattern to match (for example, \b\d{3}-\d{2}-\d{4}\b for US SSNs).
  3. Matches are always replaced with the fixed token [REDACTED].
  4. Redactions run before evaluators, so you can combine a redact guard with an evaluator guard on the same rule.

Tool filter guard

Blocks specific tool calls by name using glob patterns.

  1. Select Tool filter as the guard type.
  2. Add blocked tool name patterns (for example, shell_exec, file_delete, Bash(*rm*)).
  3. Patterns use * and ? wildcards. Patterns containing ( also match against tool call arguments, enabling argument-level filtering.

Configure priority and matching

  • Priority — guards run in ascending priority order. Lower numbers run first.
  • Selector — determines which requests the guard applies to. Default is all.
  • Match filters — narrow the guard to specific agents, models, or tags (for example, agent_name: ["my-agent"] or model.provider: ["openai"]).

Configure your SDK

Enable hooks in your SDK configuration to start calling guards:

SettingDefaultDescription
hooks.enabledfalseEnable guard evaluation.
hooks.endpointAgent Observability URLOverride the hooks endpoint.
hooks.timeoutMs5000Client-side timeout in milliseconds.
hooks.failOpentrueProceed with the LLM call if the guard request fails.
hooks.phases["preflight"]Which phases to evaluate.

When transformed_input is present in the guard response, the SDK uses the sanitized messages for the LLM call.

Example: Block PII and moderate content

A common setup combines a redact guard for PII redaction with an evaluator guard for content moderation:

  1. Guard 1 (priority 1): Redact guard with regex patterns for SSN, email, and phone number redaction.
  2. Guard 2 (priority 10): Evaluator guard with an LLM judge that checks whether the (now redacted) input is appropriate.

Guard 1 runs first and sanitizes the input. Guard 2 then evaluates the sanitized content. If the content is inappropriate, the request is denied.

Monitor guard activity

In Evaluations > Guards, open an evaluator or tool filter guard to review its activity over the last 24 hours. The page shows counts of matches, denials, warnings, and redactions, with a Denied conversations or Warned conversations list.

The list shows up to 100 recent decisions linked to conversations, including the agent, reason, and last activity. Click a row to inspect the conversation. You need permission to read conversations to view these links. Denials without a conversation ID still contribute to the counts, so an empty list doesn’t mean the guard hasn’t blocked requests.

Click Edit guard to change its configuration, or View guard if you have read-only access. Redact guards open their configuration directly.

Guard execution emits Prometheus metrics:

  • Request rate and latency — track how guards affect your request path latency.
  • Allow/deny rates — monitor how often guards block requests.
  • Per-rule evaluations — see which rules trigger most frequently.

Next steps