Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Documentation Grafana Pyroscope Configure the client to send profiles Grafana Alloy and Grafana Agent

Grafana Alloy and Grafana Agent

You can send data from your application using Grafana Alloy or Grafana Agent collectors. Both collectors support profiling with eBPF, Java, and Golang in pull mode.

Grafana Alloy is a vendor-neutral distribution of the OpenTelemetry (OTel) Collector. Alloy uniquely combines the very best OSS observability signals in the community. Grafana Alloy uses configuration file written using River.

Alloy is the recommended collector instead of Grafana Agent. New installations should use Alloy.

Grafana Agent is a powerful tool for collecting and forwarding profiling data. With the introduction of support for eBPF and continuing support for Golang in pull mode, Grafana Agent has become even more versatile in its capabilities. This document provides an overview of these two modes of profiling and guides users on setting them up.

eBPF profiling

eBPF (Extended Berkeley Packet Filter) is a modern Linux kernel technology that allows for safe, efficient, and customizable tracing of system and application behaviors without modifying the source code or restarting processes.

Benefits of eBPF profiling:

• Low overhead: eBPF collects data with minimal impact on performance.
• Versatile: eBPF can trace system calls, network packets, and even user-space application logic.
• Dynamic: No need to recompile or restart applications. eBPF allows for live tracing.

Set up eBPF profiling

1. Ensure your system runs a Linux kernel version 4.9 or newer.
2. Install a collector, such as Grafana Alloy (preferred) or Grafana Agent (legacy), on the target machine or container.
3. Configure the Agent to use eBPF for profiling. Refer to the eBPF documentation for detailed steps.
4. The collector collects eBPF profiles and sends them to the Pyroscope server.

Supported languages

This eBPF profiler only collects CPU profiles. Generally, natively compiled languages like C/C++, Go, and Rust are supported. Refer to Troubleshooting unknown symbols for additional requirements.

Python is the only supported high-level language, as long as python_enabled=true. Other high-level languages like Java, Ruby, PHP, and JavaScript require additional work to show stack traces of methods in these languages correctly. Currently, the CPU usage for these languages is reported as belonging to the runtime’s methods.

Golang profiling in pull mode

In pull mode, the collector periodically retrieves profiles from Golang applications, specifically targeting the pprof endpoints.

Benefits of Golang profiling in pull mode

• Non-intrusive: No need to modify your application’s source code.
• Centralized profiling: Suitable for environments with multiple Golang applications or microservices.
• Automatic: The agent handles the pulling and sending of profiles, requiring minimal configuration.

Set up Golang profiling in pull mode

1. Ensure your Golang application exposes pprof endpoints.
2. Install and configure the collector, either Alloy or Agent, on the same machine or container where your application runs.
3. Ensure the collector is set to pull mode and targeting the correct pprof endpoints. For step-by-step instructions, visit the Go (Pull Mode) documentation.
4. The collector queries the pprof endpoints of your Golang application, collects the profiles, and forwards them to the Pyroscope server.

Next steps

Whether using eBPF for versatile system and application profiling or relying on Golang’s built-in pprof endpoints in pull mode, Grafana Agent and Grafana Alloy collectors offer streamlined processes to gather essential profiling data. Choose the method that best fits your application and infrastructure needs.