Observability (metrics, tracing, and logs)

The MCP server can expose Prometheus metrics and supports OpenTelemetry distributed tracing and log export, following the OTel MCP semantic conventions.

Metrics require the SSE or streamable-http transport. Tracing and log export use standard OTEL_* environment variables and work with any transport, independently of --metrics.

Note: mcp-grafana currently only supports the OTLP/gRPC transport for both traces and logs. OTEL_EXPORTER_OTLP_PROTOCOL (and its _TRACES_PROTOCOL / _LOGS_PROTOCOL variants) are not honored — gRPC is used regardless.

What you’ll achieve

You can scrape MCP operation metrics (HTTP transports only) and export traces and logs to Tempo, Loki, or Grafana Cloud under any transport, including stdio.

Before you begin

The server running with SSE or streamable-http (metrics are not available with stdio).

Enable Prometheus metrics

When using SSE or streamable HTTP transports, enable Prometheus metrics with --metrics:

# Metrics on the main server at /metrics
./mcp-grafana -t streamable-http --metrics

# Metrics on a separate listen address
./mcp-grafana -t streamable-http --metrics --metrics-address :9090

Available metrics:

Metric	Type	Description
`mcp_server_operation_duration_seconds`	Histogram	MCP operation duration (labels: `mcp_method_name`, `gen_ai_tool_name`, `error_type`, `network_transport`, `mcp_protocol_version`, and — for `tools/call` on selected tools — `mcp_tool_operation`, `mcp_tool_resource_type`, `mcp_tool_phase`)
`mcp_server_session_duration_seconds`	Histogram	MCP client session duration (labels: `network_transport`, `mcp_protocol_version`)
`http_server_request_duration_seconds`	Histogram	HTTP server request duration (from otelhttp)

Note: Metrics are only available when using SSE or streamable HTTP transports. They are not available with stdio transport.

Tool-call dimension labels

For tools/call, mcp_server_operation_duration_seconds can carry up to three extra low-cardinality labels so durations are sliceable by what the call was doing:

Label	Source	Notes
`mcp_tool_operation`	the tool’s `operation` argument	Multiplexer tools only (e.g. `alerting_manage_rules`); one of the tool’s declared operations, else `other`.
`mcp_tool_resource_type`	the tool’s `type` argument	e.g. the datasource plugin type on `create_datasource`; a plugin type the server ships a schema for, else `other`.
`mcp_tool_phase`	the tool’s result `_meta`	Phase of a multi-call flow (e.g. `create_datasource` schema guidance vs. actual creation).

Arguments are raw client input, so the two argument-derived labels are allowlisted by tool and by value: unlisted tools emit neither, and unlisted values collapse into one other bucket, giving each label a fixed maximum series count. Tool validation alone would not bound them — a rejected operation is still instrumented, and create_datasource’s type is free text that succeeds for unknown values. mcp_tool_phase is exempt: the tool sets it on its own result, and tools must keep that value set small.

The high-cardinality target of a call (the datasource uid, else name) is never a metric label — it is span-only as mcp.tool.target, and empty for calls that name no entity (see below).

Enable OpenTelemetry tracing

When OTEL_EXPORTER_OTLP_ENDPOINT (or the signal-specific OTEL_EXPORTER_OTLP_TRACES_ENDPOINT) is set, the server exports traces via OTLP/gRPC.

Local example:

# Send traces to a local Tempo instance
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 \
OTEL_EXPORTER_OTLP_INSECURE=true \
./mcp-grafana -t streamable-http

Grafana Cloud example:

# Send traces to Grafana Cloud with authentication
OTEL_EXPORTER_OTLP_ENDPOINT=https://tempo-us-central1.grafana.net:443 \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic ..." \
./mcp-grafana -t streamable-http

Tool call spans follow naming like tools/call <tool_name> and include attributes such as gen_ai.tool.name, mcp.method.name, and mcp.session.id. The server supports W3C trace context propagation from the _meta field of tool call requests.

Tool-call spans also carry the tool-call dimensions mcp.tool.operation, mcp.tool.resource_type, and mcp.tool.phase, plus the span-only mcp.tool.target (the datasource uid, else name) for grouping the spans that touch one entity. mcp.tool.target is empty when a call names no entity — notably create_datasource’s phase=schema call, where the datasource does not exist yet — so stitching that call to the later phase=created one relies on the shared trace (when the client propagates context via _meta) or on mcp.session.id plus mcp_tool_resource_type, rather than on the target. Unlike the metric labels, these are attached for all tools with their raw values (no allowlist, no other bucket), since traces are high-cardinality by design. They require an HTTP transport (SSE or streamable-http, so a server span exists to enrich) and tracing enabled, but not --metrics; with stdio or tracing off, enrichment is a no-op.

Enable OpenTelemetry logs

When OTEL_EXPORTER_OTLP_ENDPOINT (or the signal-specific OTEL_EXPORTER_OTLP_LOGS_ENDPOINT) is set, the server also exports structured logs via OTLP/gRPC in addition to the existing plain-text stderr output. Logs carry trace_id and span_id from the active span so they correlate with exported traces.

Traces and logs resolve their endpoints independently, so the two signals can be enabled separately:

Setting only OTEL_EXPORTER_OTLP_TRACES_ENDPOINT enables tracing without log export.
Setting only OTEL_EXPORTER_OTLP_LOGS_ENDPOINT enables log export without tracing.
Setting the generic OTEL_EXPORTER_OTLP_ENDPOINT enables both.

# Send logs and traces to a local OTel collector
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 \
OTEL_EXPORTER_OTLP_INSECURE=true \
./mcp-grafana -t streamable-http

Stderr logging continues unchanged; operators can pipe stderr to /dev/null if they only want logs going to the OTel collector.

Logs can be sent directly to any managed backend that accepts OTLP/gRPC — for example, Grafana Cloud — by pointing OTEL_EXPORTER_OTLP_LOGS_ENDPOINT (or the generic OTEL_EXPORTER_OTLP_ENDPOINT) at the remote gRPC endpoint and supplying auth via OTEL_EXPORTER_OTLP_LOGS_HEADERS (or OTEL_EXPORTER_OTLP_HEADERS), mirroring the tracing example above. A local OTel collector is optional — useful for fan-out, batching, or multi-backend routing, but not required.

The signal-specific variants OTEL_EXPORTER_OTLP_LOGS_ENDPOINT, OTEL_EXPORTER_OTLP_LOGS_HEADERS, OTEL_EXPORTER_OTLP_LOGS_INSECURE, OTEL_EXPORTER_OTLP_LOGS_CERTIFICATE, OTEL_EXPORTER_OTLP_LOGS_TIMEOUT, and OTEL_EXPORTER_OTLP_LOGS_COMPRESSION are honored and override their generic OTEL_EXPORTER_OTLP_* counterparts — see the OTel exporter spec for the full list and precedence rules.

Note: If the configured collector is unreachable, log records are buffered in memory (default queue: 2048) and the oldest records are dropped once the queue fills. The process continues without blocking the service. Configure a local OTel collector if you need lossless buffering during outages.

Logs are also exported under the stdio transport, which makes it easy to centralize logs from local mcp-grafana instances invoked by IDE clients.

Run with Docker (metrics, tracing, and logs)

docker run --rm -p 8000:8000 \
  -e GRAFANA_URL=http://localhost:3000 \
  -e GRAFANA_SERVICE_ACCOUNT_TOKEN=<your token> \
  -e OTEL_EXPORTER_OTLP_ENDPOINT=http://tempo:4317 \
  -e OTEL_EXPORTER_OTLP_INSECURE=true \
  grafana/mcp-grafana \
  -t streamable-http --metrics