What's new in Pyroscope: profile more of your workflows, troubleshoot with AI agents, and more

When an application slows down or memory usage keeps climbing, knowing which code is responsible can help you find a fix faster. Continuous profiling gives you that visibility, so you can see where your application spends resources such as CPU time and memory.

We’ve made updates to Pyroscope, our open source continuous profiling database, that make it easier to profile your applications, find performance bottlenecks, and optimize your code, whether you're doing the work yourself or with an AI agent. 

Highlights include:

• Profile more of your workloads: Memory profiling for Python and Rust, and .NET support on Windows, helps you find performance bottlenecks across more of your services.
• Let AI agents work with your profiling data: Pyroscope commands in gcx, the Grafana CLI, let an agent query your profiles directly.
• See more of your profiling data in Grafana: Exemplars and a new Call Tree view help you explore your profiles visually, whether you're investigating on your own or looking at what an agent found.

Many of these capabilities build directly on Pyroscope 2.0, the ground-up rearchitecture of Pyroscope that makes continuous profiling more cost-effective at scale (in our production environment, it reduced the symbol storage footprint by up to 95%). For example, exemplars require the v2 storage layer that Pyroscope 2.0 is built on. 

Read on to learn more about some of the latest updates. And for a full list of recent changes, see the release notes for Pyroscope and Profiles Drilldown.

Profile more of your workloads

Memory profiling and Windows support have been some of the most frequently requested features for Pyroscope. 

If you run Python or Rust services, you can now use the Pyroscope SDKs to see where their memory goes, not just their CPU time. If you run .NET applications on Windows, you can now profile those, too.

Find memory leaks and heavy allocation: memory profiling for Python

Memory issues can be hard to track down from metrics alone. A container's memory graph shows that usage is climbing, but not which code is allocating that memory. CPU profiles don't answer that either, since they show where your code spends time. Memory profiles show which code paths allocate the most memory and which allocated the memory that's still in use, which is what you need when a service's memory keeps growing, or its allocations drive up garbage collection work.

The Python SDK now supports memory profiling, so you can see which code allocates the most memory and which code allocated the memory that's still in use, alongside the CPU profiles you already collect. Enable it with mem_enabled=True:

Copy

pyroscope.configure(
    application_name = "my.python.app",
    server_address   = "http://my-pyroscope-server:4040",
    mem_enabled      = True,
)

To learn more, see the PR and the Python SDK docs.

Find memory leaks and heap growth: memory profiling for Rust

In addition to CPU profiling, the Pyroscope Rust SDK now supports memory profiling through the jemalloc allocator. It reports in-use memory, so you can see which code paths allocated the most live memory and track down heap growth and leaks.

To learn more, see the PR, the Rust SDK docs, and the jemalloc example.

Profile .NET applications on Windows 

If you run .NET applications on Windows, including .NET Framework applications hosted in IIS, you can now send profiles from them to Pyroscope and find hot spots without migrating them first. 

Windows support is for x64, starting with version 1.3.0 of the profiler. On .NET 8, 9, and 10, you get the same profile types on Windows as on Linux, enabled with the same environment variables. On .NET Framework 4.7 and later, CPU and wall-time profiling are available.

To learn more, see the PR and the .NET SDK docs. Note: this feature is currently in public preview. 

Let AI agents work with your profiling data

Without access to your profiles, an AI coding agent can only reason about why a service is slow from the code in front of it, or you have to copy flame graphs and numbers into the conversation by hand. With access to your profiles, it can work from the same data you would, with visibility into which services use the most resources, which functions are responsible, and what a specific profile looked like at a specific time.

gcx, the Grafana CLI, gives you and your AI coding agent structured access to your Grafana instance, from dashboards and alerts to metrics, logs, traces, and profiles. gcx enables agent mode automatically in many coding agents, such as Claude Code, GitHub Copilot, and Cursor, and you can enable it yourself for any other agent with --agent. 

gcx now also includes gcx profiles commands for querying Pyroscope. To see gcx in action with Pyroscope, both in the terminal and with an AI agent, watch this community call.

Below are some of the ways you can use gcx with your profiling data, whether you run the commands yourself or an agent runs them for you. You don't need to run your own stack to try them. Grafana Play has demo data you can query, so you can point gcx at it by adding a play context:

Copy

gcx config set stacks.play.grafana.server https://play.grafana.org
gcx config set stacks.play.grafana.org-id 1
gcx config set contexts.play.stack play
export GCX_CONTEXT=play

GCX_CONTEXT selects the play context for commands in this terminal session, without changing your default context. The commands below also use the Play profiles data source, grafanacloud-profiles. 

To use your own data, open a new terminal, leave out -d grafanacloud-profiles, and change the service name.

Start with the services using the most resources

When you run many services, it's not always clear which one to look at first. gcx profiles metrics --top ranks your services by a profile type, such as CPU, over a time window, so you or your agent can start with the ones using the most resources:

Copy

gcx profiles metrics '{}' \
  -d grafanacloud-profiles \
  --profile-type process_cpu:cpu:nanoseconds:cpu:nanoseconds \
  --since 1h \
  --top

To try this with your AI coding agent and the Grafana Play data, use this prompt:

Copy

Using gcx with the play context, find which services used the most CPU over the last hour in the grafanacloud-profiles datasource.

To learn more, see the PR and the gcx profiles metrics CLI reference docs.

Go from the combined view to the exact profile with exemplars

An aggregated profile blends every instance of a service together over the time range, so a single misbehaving process can be hard to spot. Exemplars show individual profiles ranked by their value for the profile type you query, such as CPU time or memory, so you or your agent can go straight to the most expensive ones and see what happened in each, instead of only the combined total.

gcx profiles exemplars lists the most expensive individual profiles, and you can pass one of their profile IDs to gcx profiles query to see the top functions in just that profile:

Copy

gcx profiles exemplars profile '{service_name="checkoutservice"}' \
  -d grafanacloud-profiles \
  --profile-type process_cpu:cpu:nanoseconds:cpu:nanoseconds \
  --since 1h

Copy

gcx profiles query '{service_name="checkoutservice"}' \
  -d grafanacloud-profiles \
  --profile-type process_cpu:cpu:nanoseconds:cpu:nanoseconds \
  --since 1h \
  --profile-id <PROFILE_ID>

If your profiles are linked to traces, you can also narrow a query to a single span or trace to see the code that ran during a slow operation or request.

To try this with your AI coding agent and the Grafana Play data, use this prompt:

Copy

Use exemplars to find the most expensive CPU profile for checkoutservice in the grafanacloud-profiles datasource in the last hour, and summarize its most CPU-intensive functions.

To learn more, see the PRs for exemplars, profile IDs, and span and trace selectors. Also see the exemplars CLI reference docs and query CLI reference docs.

Export profiles to use with other tools

Sometimes you want the raw profile: to open it in tools you already use, to attach it to an issue, or to keep a snapshot from before a change so you can compare later. Add -o pprof to gcx profiles query to download the profile as a pprof file, which you can open with go tool pprof.

You can also start from the Grafana UI. With the release of Profiles Drilldown 3.0 in October, you can click the Export profile data download icon above the flame graph and select gcx command, which copies a gcx profiles query command that downloads the profile you're looking at as a pprof file. In the diff flame graph view, gcx commands copies two commands, one for the baseline profile and one for the comparison.

To learn more, see the PRs for -o pprof and copying gcx commands, the Export profile data in the Profiles Drilldown docs, and the gcx profiles query CLI reference docs.

See more of your profiling data in Grafana

The Grafana UI now includes exemplars to look at one profile at a time, and the Call Tree, to follow exactly how your code reached an expensive function. Use them on your own, or to look at what an agent found and see the same data for yourself.

See one profile at a time with exemplars

The same individual profiles that gcx profiles exemplars returns are also in the Grafana UI. When you're looking at a spike, exemplars let you see the individual profiles from single processes at that moment, instead of only the combined total for the whole time range. 

Exemplars show up as markers on the time series, plotted at each profile's value. In Profiles Drilldown, they're on by default in the Flame graph and Labels views, and you can turn them on in other views. Click a marker to see its labels, such as the host or, if you run on Kubernetes, the pod and namespace. Then select View profile to see the flame graph for just the profile. In Explore, turn on Exemplars under the Pyroscope query's Options.

To learn more, see our docs on exemplars concepts, drilling down to an individual profile, and available views, as well as this community call.

Follow call paths with the Call Tree view

Flame graphs and the top table are good at showing where resources are spent, but in deep or wide stacks, it can be hard to follow the exact path your code took to reach an expensive function. The new Call Tree view shows the same data as an expandable tree, so you can follow that path one call at a time. It's available in Profiles Drilldown 2.3.0 and later, in Explore, and in the flame graph panel starting in Grafana 13.3. 

On earlier Grafana versions, enable the flameGraphWithCallTree feature toggle.

When an expensive function, such as a serialization or logging call, is called from many places, use Show callers to see every path that leads to it and which caller matters most. You can also show the Call Tree side by side with a flame graph, and both stay in sync as you focus on a function.

To learn more, see the PR, the call tree mode docs, and the August community call.

How to learn more

To see the full list of recent improvements and fixes, please refer to the release notes for Pyroscope and Profiles Drilldown, and the gcx repository.

If you are interested in hearing more about Pyroscope news, please join us on the Grafana Labs Community Slack or start a conversation in GitHub Discussions. We'd love to hear what you think.

Miss out on ObservabilityCON 2026? Check out our blog post recapping all the big announcements!