Link tracing and profiling with Span Profiles on Grafana Labs

Span profiles with Traces to profiles for Go

Wed, 08 Apr 2026 14:38:28 +0000

Span profiles with Traces to profiles for Go

Span Profiles represent a major shift in profiling methodology, enabling deeper analysis of both tracing and profiling data. Traditional continuous profiling provides an application-wide view over fixed intervals. In contrast, Span Profiles delivers focused, dynamic analysis on specific execution scopes within applications, such as individual requests or specific trace spans.

This shift enables a more granular view of performance, enhancing the utility of profiles by linking them directly with traces for a comprehensive understanding of application behavior. As a result, engineering teams can more efficiently identify and address performance bottlenecks.

To learn more about Span Profiles, refer to Combining tracing and profiling for enhanced observability: Introducing Span Profiles.

Pyroscope can integrate with distributed tracing systems supporting the OpenTelemetry standard. This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans.

Note

Only CPU profiling is supported.

Because of how sampling profilers work, spans shorter than the sample interval may not be captured. Go CPU profiler probes stack traces 100 times per second, meaning that spans shorter than 10ms may not be captured.

To use Span Profiles, you need to:

Configure Pyroscope to send profiling data
Configure a client-side package to link traces and profiles: Go
Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles

Before you begin

Your applications must be instrumented for profiling and tracing before you can use span profiles.

Profiling: Your application must be instrumented with Pyroscope’s Go SDK. If you haven’t done this yet, please refer to the Go (push mode) guide.
Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven’t done this yet, please refer to the OpenTelemetry guide.

Configure the `otel-profiling-go` package

To start collecting Span Profiles for your Go application, you need to include the otel-profiling-go package in your code.

This package is a TracerProvider implementation that labels profiling data with span IDs. This makes it possible to query for span-specific profiling data with a Tempo data source configured in Grafana or Grafana Cloud.

# Make sure you also upgrade pyroscope server to version 0.14.0 or higher.
go get github.com/grafana/otel-profiling-go

Next, you need to create and configure the tracer provider:

package main

import (
	otelpyroscope "github.com/grafana/otel-profiling-go"
	"github.com/grafana/pyroscope-go"
)

func main() {
	// Initialize your tracer provider as usual.
	tp := initTracer()

	// Wrap it with otelpyroscope tracer provider.
	otel.SetTracerProvider(otelpyroscope.NewTracerProvider(tp))

	// If you're using Pyroscope Go SDK, initialize pyroscope profiler.
	_, _ = pyroscope.Start(pyroscope.Config{
		ApplicationName: "my-service",
		ServerAddress:   "http://localhost:4040",
	})

	// Your code goes here.
}

Now that you set up the tracer, you can create a new trace from anywhere and the profiler automatically captures profiles for it.

ctx, span := otel.Tracer("tracerName").Start(ctx, "ExampleSpan")
defer span.End()

// Your code goes here.

View the span profiles in Grafana or Grafana Cloud

To view the span profiles in Grafana Tempo, you need to have a Grafana instance with a Tempo data source configured to link trace spans and profiles. Refer to the configuration documentation for Grafana or Grafana Cloud.

To learn how to set up Traces to profiles and view the span profiles, refer to Traces to profiles.

Examples

Check out the examples directory for a complete demo application that shows tracing integration features.

Span profiles with Traces to profiles for Java

Wed, 08 Apr 2026 14:38:28 +0000

Span profiles with Traces to profiles for Java

To learn more about Span Profiles, refer to Combining tracing and profiling for enhanced observability: Introducing Span Profiles.

Pyroscope integrates with distributed tracing systems supporting the OpenTelemetry standard. This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans.

Note

Only CPU profiling is supported at the moment.

Because of how sampling profilers work, spans shorter than the sample interval may not be captured.

To use Span Profiles, you need to:

Configure Pyroscope to send profiling data
Configure a client-side package to link traces and profiles: Java
Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles

Before you begin

Your applications must be instrumented for profiling and tracing before you can use span profiles.

Profiling: Your application must be instrumented with Pyroscope’s Java client SDK. Refer to the Java guide for instructions.
Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the OpenTelemetry guide for instructions.

Configure the otel-profiling-java package

To start collecting Span Profiles for your Java application, you need to include otel-profiling-java as an extension for the OTel Java agent.

Assuming you have this sample application Docker image:

# [...]

EXPOSE 5000

CMD ["java", "-Dserver.port=5000", "-jar", "./my-app.jar" ]

By adding the OTel Java agent and the Pyroscope OTel Java Agent extension, you can enrich your profiles with span IDs. This makes it possible to query for span-specific profiling data in Grafana Tempo:

# [...]

EXPOSE 5000

## Add required libararies
ADD https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.17.0/opentelemetry-javaagent.jar opentelemetry-javaagent.jar
ADD https://github.com/grafana/otel-profiling-java/releases/download/v1.0.4/pyroscope-otel.jar pyroscope-otel.jar

ENV PYROSCOPE_APPLICATION_NAME=my-app
ENV PYROSCOPE_FORMAT=jfr
ENV PYROSCOPE_PROFILING_INTERVAL=10ms
ENV PYROSCOPE_PROFILER_EVENT=itimer
ENV PYROSCOPE_PROFILER_LOCK=10ms
ENV PYROSCOPE_PROFILER_ALLOC=512k
ENV PYROSCOPE_UPLOAD_INTERVAL=15s
ENV OTEL_JAVAAGENT_EXTENSIONS=./pyroscope-otel.jar
ENV OTEL_PYROSCOPE_ADD_PROFILE_URL=false
ENV OTEL_PYROSCOPE_ADD_PROFILE_BASELINE_URL=false
ENV OTEL_PYROSCOPE_START_PROFILING=true

## Useful for debugging
# ENV PYROSCOPE_LOG_LEVEL=debug

## Those environment variables need to be overwritten at runtime, if you are using Grafana Cloud
ENV PYROSCOPE_SERVER_ADDRESS=http://localhost:4040
# ENV PYROSCOPE_BASIC_AUTH_USER=123     ## Grafana Cloud Username
# ENV PYROSCOPE_BASIC_AUTH_PASSWORD=glc_secret ## Grafana Cloud Password / API Token

## Add the opentelemetry java agent
CMD ["java", "-Dserver.port=5000", "-javaagent:./opentelemetry-javaagent.jar", "-jar", "./my-app.jar" ]

Available configuration options

Flag	Description	Default
`otel.pyroscope.start.profiling`	Boolean flag to start PyroscopeAgent. Set to false if you want to start the PyroscopeAgent manually.	`true`
`otel.pyroscope.root.span.only`	Boolean flag. When enabled, the tracer will annotate only the first span created locally (the root span), but the profile will include samples of all the nested spans. This may be helpful in case if the trace consists of multiple spans shorter than 10ms and profiler can’t collect and annotate samples properly.	`true`
`otel.pyroscope.add.span.name`	Boolean flag. Controls whether the span name added to profile labels.	`true`

View the span profiles in Grafana Tempo

To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link trace spans and profiles.

Refer to the data source configuration documentation to see how to configure the visualization to link trace spans with profiles.

To use a simple configuration, follow these steps:

Select a Pyroscope data source from the Data source drop-down.
Optional: Choose any tags to use in the query. If left blank, the default values of service.name and service.namespace are used.

The tags you configure must be present in the spans attributes or resources for a trace to profiles span link to appear. You can optionally configure a new name for the tag. This is useful for example if the tag has dots in the name and the target data source doesn’t allow using dots in labels. In that case you can for example remap service.name to service_name.

Select one or more profile types to use in the query. Select the drop-down and choose options from the menu.

The profile type or app must be selected for the query to be valid. Grafana doesn’t show any data if the profile type or app isn’t selected when a query runs.

Examples

Check out the examples directory for a complete demo application that shows tracing integration features.

Span profiles with Traces to profiles for Ruby

Wed, 08 Apr 2026 14:38:28 +0000

Span profiles with Traces to profiles for Ruby

To learn more about Span Profiles, refer to Combining tracing and profiling for enhanced observability: Introducing Span Profiles.

Note

Only CPU profiling is supported at the moment.

Because of how sampling profilers work, spans shorter than the sample interval may not be captured.

To use Span Profiles, you need to:

Configure Pyroscope to send profiling data
Configure a client-side package to link traces and profiles: Ruby
Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles

Before you begin

Your applications must be instrumented for profiling and tracing before you can use span profiles.

Profiling: Your application must be instrumented with Pyroscope’s Ruby SDK. Refer to the Ruby guide for instructions.
Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the OpenTelemetry guide for instructions.

Configure the otel-profiling-ruby package

To start collecting Span Profiles for your Ruby application, you need to include otel-profiling-ruby in your code.

This package is a TracerProvider implementation, that labels profiling data with span IDs which makes it possible to query for span-specific profiling data in Grafana Tempo UI.

# Add to your Gemfile
gem install pyroscope-otel

Next, you need to create and configure the tracer provider:

Pyroscope.configure do |config|
  # Configure pyroscope as described https://pyroscope.io/docs/ruby/
end

OpenTelemetry::SDK.configure do |config|
  config.add_span_processor Pyroscope::Otel::SpanProcessor.new(
    "#{app_name}.cpu", # your app name with ".cpu" suffix, for example rideshare-ruby.cpu
    pyroscope_endpoint # link to your pyroscope server, for example "http://localhost:4040"
  )
  # Configure the rest of opentelemetry as described  https://github.com/open-telemetry/opentelemetry-ruby
end

View the span profiles in Grafana Tempo

To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link trace spans and profiles.

Refer to the data source configuration documentation to see how to configure the visualization to link trace spans with profiles.

To use a simple configuration, follow these steps:

Select a Pyroscope data source from the Data source drop-down.
Optional: Choose any tags to use in the query. If left blank, the default values of service.name and service.namespace are used.

Select one or more profile types to use in the query. Select the drop-down and choose options from the menu.

The profile type or app must be selected for the query to be valid. Grafana doesn’t show any data if the profile type or app isn’t selected when a query runs.

Examples

Check out the examples directory for a complete demo application that shows tracing integration features.

Examples

Check out these demo applications for span profiles:

Ruby example
Other examples in multiple languages

Span profiles with Traces to profiles for .NET

Wed, 08 Apr 2026 14:38:28 +0000

Span profiles with Traces to profiles for .NET

Span Profiles represent a shift in profiling methodology. Traditional continuous profiling provides an application-wide view over fixed intervals. In contrast, Span Profiles delivers focused, dynamic analysis on specific execution scopes within applications, such as individual requests or specific trace spans.

To learn more about Span Profiles, refer to Combining tracing and profiling for enhanced observability: Introducing Span Profiles.

Supported profile types

Note

Only CPU profiling is supported at the moment.

Because of how sampling profilers work, spans shorter than the sample interval may not be captured.

For a more detailed list of supported profile types, refer to Profile types.

Before you begin

To use Span Profiles, you need to:

Configure Pyroscope to send profiling data
Configure a client-side package to link traces and profiles: .NET
Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles

Instrument your application for profiles

Your applications must be instrumented for profiling and tracing before you can use span profiles.

Profiling: Your application must be instrumented with Pyroscope’s .NET instrumentation library. Refer to the .NET guide for instructions.
Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the OpenTelemetry guide for instructions.

Note
Span profiles in .NET are only supported using OpenTelemetry manual instrumentation because Pyroscope’s .NET profiler and OpenTelemetry’s auto instrumentation are based on separate .NET CLR profilers.

Configure the `Pyroscope.OpenTelemetry` package

To start collecting Span Profiles for your .NET application, you need to include Pyroscope.OpenTelemetry in your code.

This package provides a SpanProcessor implementation, which connects the two telemetry signals (traces and profiles) together.

dotnet add package Pyroscope.OpenTelemetry

Next, create and register the PyroscopeSpanProcessor:

builder.Services.AddOpenTelemetry()
    .WithTracing(b =>
    {
        b
        .AddAspNetCoreInstrumentation()
        .AddConsoleExporter()
        .AddOtlpExporter()
        .AddProcessor(new Pyroscope.OpenTelemetry.PyroscopeSpanProcessor());
    });

With the span processor registered, spans created automatically (for example, HTTP handlers) and manually (ActivitySource.StartActivity()) have profiling data associated with them.

View the span profiles in Grafana

To view the span profiles in Grafana Explore or Grafana Traces Drilldown, you need to have a Grafana instance running and a Tempo data source configured to link traces and profiles.

Refer to the Tempo data source configuration documentation to see how to configure the visualization to link traces with profiles.

Examples

Check out these demo applications for span profiles:

.NET example
Other examples in multiple languages

Span profiles with Traces to profiles for Python

Wed, 08 Apr 2026 14:38:28 +0000

Span profiles with Traces to profiles for Python

To learn more about Span Profiles, refer to Combining tracing and profiling for enhanced observability: Introducing Span Profiles.

Note

Only CPU profiling is supported at the moment.

Because of how sampling profilers work, spans shorter than the sample interval may not be captured.

To use Span Profiles, you need to:

Configure Pyroscope to send profiling data
Configure a client-side package to link traces and profiles: Python
Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles

Before you begin

Your applications must be instrumented for profiling and tracing before you can use span profiles.

Profiling: Your application must be instrumented with Pyroscope’s Python instrumentation library. Refer to the Python guide for instructions.
Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the OpenTelemetry guide for isntructions.

Configure the `pyroscope-otel` package

To start collecting Span Profiles for your Python application, you need to include pyroscope-otel in your code.

This package provides a SpanProcessor implementation, which connects the two telemetry signals (traces and profiles) together.

pip install pyroscope-otel

Next, create and register the PyroscopeSpanProcessor:

# import span processor
from pyroscope.otel import PyroscopeSpanProcessor

# obtain a OpenTelemetry tracer provider
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
provider = TracerProvider()

# register the span processor
provider.add_span_processor(PyroscopeSpanProcessor())

# register the tracer provider
trace.set_tracer_provider(provider)

With the span processor registered, spans created automatically (for example, HTTP handlers) and manually will have profiling data associated with them.

View the span profiles in Grafana Tempo

To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link traces and profiles.

Refer to the data source configuration documentation to see how to configure the visualization to link traces with profiles.

Examples

Check out these demo applications for span profiles:

Python example
Other examples in multiple languages

Link tracing and profiling with Span Profiles on Grafana Labs

Span profiles with Traces to profiles for Go

Span profiles with Traces to profiles for Go

Before you begin

Configure the otel-profiling-go package

View the span profiles in Grafana or Grafana Cloud

Examples

Span profiles with Traces to profiles for Java

Span profiles with Traces to profiles for Java

Before you begin

Configure the otel-profiling-java package

Available configuration options

View the span profiles in Grafana Tempo

Examples

Span profiles with Traces to profiles for Ruby

Span profiles with Traces to profiles for Ruby

Before you begin

Configure the otel-profiling-ruby package

View the span profiles in Grafana Tempo

Examples

Examples

Span profiles with Traces to profiles for .NET

Span profiles with Traces to profiles for .NET

Supported profile types

Before you begin

Instrument your application for profiles

Configure the Pyroscope.OpenTelemetry package

View the span profiles in Grafana

Examples

Span profiles with Traces to profiles for Python

Span profiles with Traces to profiles for Python

Before you begin

Configure the pyroscope-otel package

View the span profiles in Grafana Tempo

Examples

Configure the `otel-profiling-go` package

Configure the `Pyroscope.OpenTelemetry` package

Configure the `pyroscope-otel` package