Documentationbreadcrumb arrow Grafana Cloudbreadcrumb arrow Observe and actbreadcrumb arrow Agent Observabilitybreadcrumb arrow Observe and troubleshootbreadcrumb arrow Cost and performance
Grafana Cloud

Optimize cost and performance

Agent Observability captures detailed token usage, cost, and cache data for every generation. Use this data to find optimization opportunities and reduce your AI spending.

Before you begin

  • Configure your agent to export generations and OpenTelemetry metrics to Grafana Cloud. For instructions, refer to Get started with Grafana Cloud.
  • Collect representative traffic for the agents and models you want to compare.

Analyze token usage

Open Analytics and review the tokens and cost dashboard. Look for:

  • High-token agents: agents that consistently use more tokens than expected may have verbose prompts or unnecessary context.
  • Model selection: some calls may use expensive models where a cheaper alternative would produce equivalent results.
  • Reasoning tokens: if reasoning tokens are high, check whether thinking mode is enabled unnecessarily.

For recorded generation costs and observed token and request prices, refer to Use Prometheus metrics directly. These metrics let you build custom cost queries; multiplying aggregate usage by current prices can differ from recorded costs.

Improve cache efficiency

Prompt caching reduces costs by reusing previously processed prompt prefixes. Agent Observability tracks cache read and cache write tokens for each generation.

To improve cache rates:

  • Keep the system prompt stable. Changing the system prompt frequently invalidates the cache.
  • Place static content (system prompt, tool definitions) at the beginning of the message sequence.
  • Minimize dynamic content that changes between requests.

Open Usage and cost and select Cache hit rate to track cache reuse over time. Compare agents or models with Break down, then inspect the Cache hit rate column in the conversations table. Select Token type to compare cache read and cache write counts instead of a percentage.

Note

If you query the token metric yourself rather than reading the dashboard, check the gen_ai_token_semantics label first. Where it reads inclusive, the input series already contains the cache tokens, so adding cache_read to it counts them twice, and only input - cache_read - cache_write - cache_creation pays the full prompt rate.

Where the label is absent, check the provider before you add the buckets up. An absent label means nothing was declared, not that the buckets are separate: OpenAI and Gemini report an inclusive input whether or not the label is there. The dashboard handles both; a hand-written query has to choose.

Reduce tool call overhead

The tools analytics page shows tool call frequency and duration. Look for:

  • Unused tools: tools that are defined but never called add prompt tokens without value. Consider removing them.
  • Slow tools: tools with high execution time may benefit from timeouts or caching.
  • Excessive tool calls: agents that call too many tools per turn may need prompt adjustments.

Attribute cost to a team, project, or developer

For SDK instrumentation, set client tags to split cost by team, project, or environment. Use the AGENTO11Y_TAGS environment variable or the tags field in the client configuration:

Bash 
export AGENTO11Y_TAGS=team=ai,env=production

Client tags reach both the generation export and the OpenTelemetry metrics, so each key becomes a Prometheus label named agento11y_tag_<KEY>. Turn on Break down in the tokens and estimated cost charts and select that label to compare spend across tag values.

A cost breakdown by a custom label uses public list prices, not your configured model prices or long-context rates. Check the Cost shown at list prices notice before comparing these estimates with recorded totals. For details, refer to Optimize costs.

Tags that you pass on a single generation stay on that generation and never become metric labels, so they can’t drive this breakdown. For the behavior of each mechanism, refer to Client tags.

Coding-agent integrations can resolve user, repository, and branch labels automatically. For setup instructions, refer to Attribute cost to a user, repository, or branch.

Compare agent versions

When you change prompts or tools, use the agent catalog to compare the new version’s cost and performance against the previous version. Check whether the change improved quality without significantly increasing cost, or vice versa.

Next steps