Documentation Grafana Cloud Observe and act Agent Observability

Agent Observability

Agent Observability for teams running LLM agents in production. Monitor conversations, costs, quality, and performance from a single pane of glass.

Overview

Agent Observability is built on OpenTelemetry and gives teams running LLM agents in production a single place to monitor agent activity, trace conversations, track costs, and evaluate quality.

Agent Observability provides thin SDKs for Go, Python, TypeScript, Java, and .NET that capture generation data with minimal code changes. Built-in framework integrations for LangChain, LangGraph, OpenAI Agents, Vercel AI SDK, and others make instrumentation automatic.

With Agent Observability in Grafana Cloud, you can browse conversations, drill into traces, compare agent versions, configure online evaluation rules and guards, review offline experiment reports, and use built-in analytics for metrics, logs, traces, and profiles.

Review pricing, privacy, and access

Before you adopt Agent Observability, review the information you need to assess it for your organization:

• Pricing: Find the billing model, current rates, included usage, and usage reporting.
• Data handling and privacy: Review collected data, retention, data residency, and where LLM judge inference runs.
• Security and access controls: Review tenant isolation, authentication, roles, and permissions.

Use Agent Observability

Use Agent Observability to understand agent behavior from development through production. You can:

• Capture LLM generations, tool calls, workflow steps, token usage, and cost data from your agents.
• Forward coding-agent sessions from supported tools without adding SDK code to an application.
• Search and inspect conversations, then connect individual generations to OpenTelemetry traces and metrics.
• Track agent versions, prompt changes, tool usage, quality scores, latency, errors, and cost trends.
• Configure online evaluations and guards to score or gate production traffic.
• Run offline experiments to compare models, prompts, and agent versions before release.

Explore

• Introduction to Agent Observability
  Understand Agent Observability's core concepts including generation capture, conversation tracing, agent versioning, online evaluation, and the observability data model.
• Understand Agent Observability pricing
  Where to find Agent Observability pricing details and how to view usage analytics.
• Get started with Agent Observability
  Enable Agent Observability on Grafana Cloud and instrument your agents.
• Observe and troubleshoot
  Inspect conversations, compare agents, and investigate performance and costs.
• Evaluate and improve
  Score live traffic and run experiments to measure changes to your agents.
• Protect agents
  Configure guards and redact sensitive content from agent requests.
• Instrumentation
  Add framework and workflow instrumentation, tune your SDK, or migrate an existing integration.
• Privacy and access
  Review data handling, redaction, residency, and access controls before you adopt Agent Observability.
• API and SDKs
  Find SDK setup guides and API contracts for generation ingest, evaluation, guards, and experiments.

Frequently asked questions