pyroscope on Grafana Labs

pyroscope.ebpf

Wed, 07 Oct 2026 16:23:35 +0000

`pyroscope.ebpf`

pyroscope.ebpf configures an eBPF profiling job for the current host. The component forwards the collected performance profiles to the receivers you list in forward_to.

The pyroscope.ebpf component embeds the grafana/opentelemetry-ebpf-profiler which is a fork of open-telemetry/opentelemetry-ebpf-profiler.

Note
To use the pyroscope.ebpf component you must run Alloy as root and inside the host PID namespace. On Kubernetes, the simplest option is to set securityContext.privileged: true. Users who prefer least-privilege can instead grant the specific capabilities required.

Note
The profiler requires file system storage at /tmp/symb-cache to store symbol cache data. Ensure this directory is accessible and has sufficient storage space.

You can specify multiple pyroscope.ebpf components by giving them different labels, however it’s not recommended as it can lead to additional memory and CPU usage.

Required privileges

When running on Kubernetes without privileged: true, grant the following Linux capabilities and mount the kernel filesystem paths:

Capability	Purpose
`BPF`	Load and manage eBPF programs and maps.
`PERFMON`	Attach performance events and read performance counters.
`SYS_PTRACE`	Read `/proc/<pid>/` entries.
`CHECKPOINT_RESTORE`	Follow magic-links in `/proc/<pid>/map_files/*` for ELF symbol reading (kernel 5.9+; use `SYS_ADMIN` on older kernels).
`SYS_RESOURCE`	Raise `RLIMIT_MEMLOCK` for eBPF map locking.
`DAC_READ_SEARCH`	Read ELF binaries and `/proc` entries regardless of DAC permission bits.
`SYSLOG`	Read the kernel ring buffer for eBPF verifier diagnostics.

Mount /sys/kernel/tracing (on older kernel versions you might need /sys/kernel/debug instead) from the host as read-only volumes so the tracer can attach tracepoints.

Supported languages

Native code (C/C++, Rust, Zig, Go, etc. without debug symbols on host)
Broad set of high-level languages (HotSpot JVM, Python, Ruby, PHP, Node.js, V8, Perl).

Usage

pyroscope.ebpf "<LABEL>" {
  targets    = <TARGET_LIST>
  forward_to = <RECEIVER_LIST>
}

Arguments

The component configures and starts an eBPF profiling job to collect performance profiles from the current host.

You can use the following arguments with pyroscope.ebpf:

Name	Type	Description	Default	Required
`forward_to`	`list(ProfilesReceiver)`	List of receivers to send collected profiles to.		yes
`bpf_fs_root`	`string`	Root path of the BPF filesystem for pinned maps used in trace correlation.	`"/sys/fs/bpf/"`	no
`build_id_cache_size`	`int`	Deprecated (no-op), previously controlled the size of the elf file build id -> symbols table LRU cache.	`64`	no
`cache_rounds`	`int`	Deprecated (no-op), previously controlled the number of cache rounds.		no
`collect_interval`	`duration`	How frequently to collect profiles.	`"15s"`	no
`collect_kernel_profile`	`bool`	Deprecated (no-op), previously enabled collection of kernel-space profiles.	`true`	no
`collect_user_profile`	`bool`	Deprecated (no-op), previously enabled collection of user-space profiles.	`true`	no
`comm`	`string`	How the component includes the process command name in profiles. One of `none`, `label`, `stackframe`, or `both`.	`"none"`	no
`container_id_cache_size`	`int`	Deprecated (no-op), previously controlled the size of the PID -> container ID table LRU cache.	`1024`	no
`demangle`	`string`	C++ `demangle` mode. Available options are: `none`, `simplified`, `templates`, or `full`.	`"none"`	no
`dotnet_enabled`	`bool`	A flag to enable or disable .NET profiling.	`true`	no
`go_enabled`	`bool`	A flag to enable or disable Go profiling.	`true`	no
`go_table_fallback`	`bool`	Deprecated (no-op), previously enabled symbol lookup in `.sym` / `.dynsym` sections when `.gopclntab` lookup failed.	`false`	no
`hotspot_enabled`	`bool`	A flag to enable or disable HotSpot profiling.	`true`	no
`kernel_frames`	`bool`	Include kernel-space frames in collected profiles. Set to `false` to drop kernel frames from each stack trace.	`true`	no
`lazy_mode`	`bool`	Enable lazy mode to defer eBPF profiler startup until the component discovers targets.	`false`	no
`load_probe`	`bool`	Load a generic eBPF program that you can attach to a kernel or user-space hook externally.	`false`	no
`no_kernel_version_check`	`bool`	Skip the kernel version check for eBPF support.	`false`	no
`obi_process_context_enabled`	`bool`	Enable profile correlation with traces generated by components compatible with OpenTelemetry eBPF Instrumentation.	`true`	no
`off_cpu_threshold`	`float`	Probability from `0` to `1` for recording an off-CPU event. A value of `0` disables off-CPU profiling.	`0`	no
`perl_enabled`	`bool`	A flag to enable or disable Perl profiling.	`true`	no
`php_enabled`	`bool`	A flag to enable or disable PHP profiling.	`true`	no
`pid_cache_size`	`int`	Deprecated (no-op), previously controlled the size of the PID -> proc symbols table LRU cache.	`32`	no
`pid_label`	`bool`	Attach the process PID to each pprof sample as a `pid` label.	`false`	no
`pid_map_size`	`int`	Deprecated (no-op), previously controlled the size of eBPF PID map.	`2048`	no
`probe_links`	`list(string)`	List of kernel or user-space probes from which to collect stack traces.		no
`python_enabled`	`bool`	A flag to enable or disable Python profiling.	`true`	no
`ruby_enabled`	`bool`	A flag to enable or disable Ruby profiling.	`true`	no
`same_file_cache_size`	`int`	Deprecated (no-op), previously controlled the size of the elf file -> symbols table LRU cache.	`8`	no
`sample_rate`	`int`	How many times per second to collect profile samples.	`19`	no
`symbols_map_size`	`int`	Deprecated (no-op), previously controlled the size of the eBPF symbols map.	`16384`	no
`targets`	`list(map(string))`	List of process or container targets to profile.		no
`u_probe_links`	`list(string)`	Deprecated. Use `probe_links` instead.		no
`v8_enabled`	`bool`	A flag to enable or disable V8 profiling.	`true`	no
`verbose_mode`	`bool`	Enable verbose logging for the eBPF profiler.	`false`	no

Caution
Use no_kernel_version_check only when you run on an older kernel that includes backported eBPF features from a later release. When you skip this check, the profiler can’t verify that your kernel has the eBPF features it needs, and profiling may fail or produce incomplete results. Proceed at your own risk.

You must set the forward_to argument. Omitted fields take their default values.

Several arguments carry the label “Deprecated (no-op)”. These arguments once configured cache sizes and behaviors, but they no longer have any effect. Remove these arguments from your configuration.

Blocks

pyroscope.ebpf doesn’t support any blocks.

Exported fields

pyroscope.ebpf doesn’t export any fields.

Component health

pyroscope.ebpf reports itself as unhealthy if you give it an invalid configuration, or if the eBPF profiling session fails to start, for example because of missing privileges or an unsupported kernel.

Debug information

elf_cache per build id and per same file symbol tables and their sizes in symbols count.
pid_cache per process elf symbol tables and their sizes in symbols count.
targets the active targets the component tracks.

Debug metrics

The following Prometheus metrics are exposed:

Name	Type	Description
`pyroscope_ebpf_active_targets`	`gauge`	Number of active targets the component tracks.
`pyroscope_ebpf_pprofs_total`	`counter`	Number of pprof profiles collected by the eBPF component, per `service_name`.
`pyroscope_ebpf_pprof_bytes_total`	`counter`	Total number of bytes in pprof profiles collected by the eBPF component, per `service_name`.
`pyroscope_ebpf_pprof_samples_total`	`counter`	Total number of samples in pprof profiles collected by the eBPF component, per `service_name`.
`pyroscope_ebpf_profiling_sessions_failing_total`	`counter`	Number of profiling sessions failed.
`pyroscope_ebpf_profiling_sessions_total`	`counter`	Number of profiling sessions started by the eBPF component.
`pyroscope_fanout_latency`	`histogram`	Write latency for sending to direct and indirect components.

eBPF profiler internal metrics

The component also exposes internal metrics from the embedded eBPF profiler. The component emits these metrics only after recording each one at least once, so not all metrics appear on every host. The metrics carry an otel_scope_name="pyroscope.ebpf" label.

Notable metrics include:

Native unwinding

Name	Type	Description
`UnwindNativeAttempts_total`	`counter`	Unwind attempts since the previous check.
`UnwindNativeFrames_total`	`counter`	Unwound frames since the previous check.
`UnwindNativeStackDeltaStop_total`	`counter`	Number of stop stack deltas in the native unwinder (success).
`UnwindNativeSmallPC_total`	`counter`	Number of times PC held a value smaller than `0x1000`.
`UnwindErrStackLengthExceeded_total`	`counter`	Number of times the unwinder exceeded `MAX_FRAME_UNWINDS`.

Interpreter unwinding

Name	Type	Description
`UnwindPythonAttempts_total`	`counter`	Number of attempted Python unwinds.
`UnwindPythonFrames_total`	`counter`	Number of unwound Python frames.
`UnwindHotspotAttempts_total`	`counter`	Number of attempted HotSpot JVM unwinds.
`UnwindHotspotFrames_total`	`counter`	Number of unwound HotSpot JVM frames.
`UnwindRubyAttempts_total`	`counter`	Number of attempted Ruby unwinds.
`UnwindRubyFrames_total`	`counter`	Number of unwound Ruby frames.
`UnwindPHPAttempts_total`	`counter`	Number of attempted PHP unwinds.
`UnwindPHPFrames_total`	`counter`	Number of unwound PHP frames.
`UnwindPerlAttempts_total`	`counter`	Number of attempted Perl unwinds.
`UnwindPerlFrames_total`	`counter`	Number of unwound Perl frames.
`UnwindV8Attempts_total`	`counter`	Number of attempted V8 unwinds.
`UnwindV8Frames_total`	`counter`	Number of unwound V8 frames.
`UnwindDotnetAttempts_total`	`counter`	Number of attempted .NET unwinds.
`UnwindDotnetFrames_total`	`counter`	Number of unwound .NET frames.

Symbolization

Name	Type	Description
`PythonSymbolizationSuccesses_total`	`counter`	Number of successfully symbolized Python frames.
`PythonSymbolizationFailures_total`	`counter`	Number of Python frames that failed symbolization.
`HotspotSymbolizationSuccesses_total`	`counter`	Number of successfully symbolized HotSpot frames.
`HotspotSymbolizationFailures_total`	`counter`	Number of HotSpot frames that failed symbolization.
`RubySymbolizationSuccess_total`	`counter`	Number of successfully symbolized Ruby frames.
`RubySymbolizationFailure_total`	`counter`	Number of Ruby frames that failed symbolization.

Process management

Name	Type	Description
`NumProcNew_total`	`counter`	Number of new PID events.
`NumProcExit_total`	`counter`	Number of exit PID events.
`NumGenericPID_total`	`counter`	Number of generic PID events.

eBPF map state

Name	Type	Description
`NumExeIDLoadedToEBPF`	`gauge`	The number of executables loaded to eBPF maps.
`HashmapPidPageToMappingInfo`	`gauge`	Current size of the `pid_page_to_mapping_info` hash map.
`HashmapNumStackDeltaPages`	`gauge`	Current size of the stack delta pages hash map.
`UnwindInfoArraySize`	`gauge`	Current size of the unwind info array.

The opentelemetry-ebpf-profiler metrics.json file defines the full list of metrics.

Profile collecting behavior

The pyroscope.ebpf component collects stack traces associated with a process running on the current host. You can use the sample_rate argument to define the number of stack traces collected per second. The default is 19.

The following labels are automatically injected into the collected profiles if you haven’t defined them. These labels can help you pin down a profiling target.

Label	Description
`__container_id__`	The container ID derived from target.
`__name__`	Pyroscope metric name. Defaults to `process_cpu`.
`service_name`	Pyroscope service name. It’s automatically selected from discovery meta labels if possible. Otherwise defaults to `unspecified`.
`otel.scope.name`	The instrumentation scope name, set to `com.grafana.alloy/pyroscope.ebpf`.
`otel.scope.version`	The Alloy version that produced the profile.

Alloy only sets otel.scope.name and otel.scope.version if they aren’t already present on the profile. otel.scope.version is only set when otel.scope.name matches the default value.

Targets

Each target in targets must include one of the following special labels, and the label must correspond to the container or process you profile:

__container_id__: The container ID.
__meta_docker_container_id: The ID of the Docker container.
__meta_kubernetes_pod_container_id: The ID of the Kubernetes Pod container.
__process_pid__ : The process ID.

Each process is then associated with a specified target from the targets list, determined by a container ID or process PID.

If a process’s container ID matches a target’s container ID label, the component aggregates the stack traces per target by container ID. If a process’s PID matches a target’s process PID label, the component aggregates the stack traces per target by process PID. Otherwise the process isn’t profiled.

Service name

Every profile carries a service_name label. You can set it on the target yourself. If you don’t, the component adds the label and infers a value, in order, from the following sources:

__meta_kubernetes_pod_annotation_pyroscope_io_service_name, which is a pyroscope.io/service_name Pod annotation.
__meta_kubernetes_namespace and __meta_kubernetes_pod_container_name, combined into ebpf/<namespace>/<container_name>.
__meta_docker_container_name.
__meta_dockerswarm_container_label_service_name.
__meta_dockerswarm_service_name.

When you don’t set service_name and the component can’t infer a value, it uses unspecified.

Troubleshoot unknown symbols

The component extracts symbols from several sources, including:

The .gopclntab section in Go language ELF files.
The .symtab and .dynsym sections in the debug ELF file.
The .symtab and .dynsym sections in the ELF file.

The search for debug files follows the GNU Debugger algorithm. For example, if the profiler wants to find the debug file for /lib/x86_64-linux-gnu/libc.so.6 with a .gnu_debuglink set to libc.so.6.debug and a build ID 0123456789abcdef. The component examines the following paths:

/usr/lib/debug/.build-id/01/0123456789abcdef.debug
/lib/x86_64-linux-gnu/libc.so.6.debug
/lib/x86_64-linux-gnu/.debug/libc.so.6.debug
/usr/lib/debug/lib/x86_64-linux-gnu/libc.so.6.debug

Deal with unknown symbols

Unknown symbols in the profiles you’ve collected indicate that the profiler couldn’t access an ELF file associated with a given address in the trace.

This can occur for several reasons:

The process has terminated, making the ELF file inaccessible.
The ELF file is either corrupted or not recognized as an ELF file.
There is no corresponding ELF file entry in /proc/pid/maps for the address in the stack trace.

Address unresolved symbols

If you only see module names without corresponding function names, for example, /lib/x86_64-linux-gnu/libc.so.6, the component couldn’t map the symbols to their respective function names.

This can occur for several reasons:

The build process stripped the binary, leaving no .symtab, .dynsym, or .gopclntab sections in the ELF file.
The debug file is missing, or the component can’t locate it.

To fix this for your binaries, ensure that they’re either not stripped or that you have separate debug files available. You can achieve this by running:

objcopy --only-keep-debug elf elf.debug
strip elf -o elf.stripped
objcopy --add-gnu-debuglink=elf.debug elf.stripped elf.debuglink

For system libraries, install the debug symbols. On Ubuntu, for example, you can install debug symbols for libc by executing:

apt install libc6-dbg

Understand flat stack traces

If your profiles show many shallow stack traces, typically 1-2 frames deep, your compiler might have omitted frame pointers.

To compile your code with frame pointers, include the -fno-omit-frame-pointer flag in your compiler options.

Examples

The following examples show how to discover profiling targets and send the resulting profiles to a Pyroscope endpoint.

Kubernetes discovery

In the following example, the component collects performance profiles from Pods on the same node, discovered using discovery.kubernetes. Pod selection relies on the HOSTNAME environment variable, which is a Pod name if Alloy is used as an Alloy Helm chart. The example sets the service_name label to ebpf/{__meta_kubernetes_namespace}/{__meta_kubernetes_pod_container_name} from Kubernetes meta labels.

discovery.kubernetes "all_pods" {
  role = "pod"
  selectors {
    field = "spec.nodeName=" + sys.env("HOSTNAME")
    role = "pod"
  }
}

discovery.relabel "local_pods" {
  targets = discovery.kubernetes.all_pods.targets
  rule {
    action = "drop"
    regex = "Succeeded|Failed"
    source_labels = ["__meta_kubernetes_pod_phase"]
  }
  rule {
    action = "replace"
    regex = "(.*)@(.*)"
    replacement = "ebpf/${1}/${2}"
    separator = "@"
    source_labels = ["__meta_kubernetes_namespace", "__meta_kubernetes_pod_container_name"]
    target_label = "service_name"
  }
  rule {
    action = "labelmap"
    regex = "__meta_kubernetes_pod_label_(.+)"
  }
  rule {
    action = "replace"
    source_labels = ["__meta_kubernetes_namespace"]
    target_label = "namespace"
  }
  rule {
    action = "replace"
    source_labels = ["__meta_kubernetes_pod_name"]
    target_label = "pod"
  }
  rule {
    action = "replace"
    source_labels = ["__meta_kubernetes_pod_node_name"]
    target_label = "node"
  }
  rule {
    action = "replace"
    source_labels = ["__meta_kubernetes_pod_container_name"]
    target_label = "container"
  }
}
pyroscope.ebpf "local_pods" {
  forward_to = [ pyroscope.write.endpoint.receiver ]
  targets = discovery.relabel.local_pods.output
}

pyroscope.write "endpoint" {
  endpoint {
    url = "http://pyroscope:4040"
  }
}

Docker discovery

The following example collects performance profiles from containers discovered by discovery.docker and ignores all other profiles collected from outside any docker container. The example sets the service_name label from the __meta_docker_container_name label.

discovery.docker "linux" {
  host = "unix:///var/run/docker.sock"
}

discovery.relabel "local_containers" {
  targets = discovery.docker.linux.targets
  rule {
    action = "replace"
    source_labels = ["__meta_docker_container_name"]
    target_label = "service_name"
  }
}

pyroscope.write "staging" {
  endpoint {
    url = "http://pyroscope:4040"
  }
}

pyroscope.ebpf "default" {
  forward_to   = [ pyroscope.write.staging.receiver ]
  targets      = discovery.relabel.local_containers.output
}

Collect profiles from probes

Use probe_links to collect stack traces whenever something calls the kernel or user-space probes you specify. Kernel probes use the format <PROBE_TYPE>:<SYMBOL>, and user-space probes use the format <PROBE_TYPE>:<EXECUTABLE_PATH>:<SYMBOL>. Supported probe types are kprobe, kretprobe, uprobe, and uretprobe.

The following example collects profiles when the kernel calls do_sys_open or when libc calls malloc:

pyroscope.ebpf "probes" {
	targets     = <TARGET_LIST>
	forward_to  = <RECEIVER_LIST>
	probe_links = [
		"kprobe:do_sys_open",
		"uprobe:/usr/lib/libc.so.6:malloc",
	]
}

Replace the following:

<TARGET_LIST>: The list of process or container targets to profile.
<RECEIVER_LIST>: The list of profile receivers to forward profiles to.

Correlate request profiles with Beyla traces and spans

When obi_process_context_enabled is true, pyroscope.ebpf uses a shared eBPF map under bpf_fs_root to correlate profile samples with traces generated by components compatible with OpenTelemetry eBPF Instrumentation (OBI), such as beyla.ebpf. Correlated samples include the trace_id and span_id labels. Components that share process context must use the same BPF filesystem root.

The following example uses beyla.ebpf to instrument requests to services listening on any port. It also profiles local processes and forwards the profiles to Pyroscope. Both components use their default BPF filesystem root at /sys/fs/bpf/.

beyla.ebpf "autoinstrument" {
	discovery {
		instrument {
			open_ports = "1-65535"
		}
	}

	output {
		traces = [otelcol.processor.batch.traces.input]
	}
}

otelcol.processor.batch "traces" {
	output {
		traces = [otelcol.exporter.otlphttp.traces.input]
	}
}

otelcol.exporter.otlphttp "traces" {
	client {
		endpoint = "<OTLP_TRACES_ENDPOINT>"
	}
}

discovery.process "all" { }

discovery.relabel "profiled_processes" {
	targets = discovery.process.all.targets

	rule {
		action        = "replace"
		source_labels = ["__meta_process_exe"]
		regex         = ".*/([^/]+)$"
		replacement   = "$1"
		target_label  = "service_name"
	}
}

pyroscope.ebpf "request_profiles" {
	targets                     = discovery.relabel.profiled_processes.output
	forward_to                  = [pyroscope.write.profiles.receiver]
	bpf_fs_root                 = "/sys/fs/bpf/"
	obi_process_context_enabled = true
}

pyroscope.write "profiles" {
	endpoint {
		url = "<PYROSCOPE_URL>"
	}
}

Replace the following:

<OTLP_TRACES_ENDPOINT>: The OTLP HTTP endpoint to send traces to.
<PYROSCOPE_URL>: The URL of the Pyroscope server to send profiles to.

Compatible components

pyroscope.ebpf can accept arguments from the following components:

Components that export Targets
Components that export Pyroscope ProfilesReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

pyroscope.enrich

Wed, 07 Oct 2026 16:23:35 +0000

`pyroscope.enrich`

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement. To enable and use an experimental component, you must set the stability.level flag to experimental.

pyroscope.enrich enriches profiles with additional labels from service discovery targets. It matches a label from incoming profiles against a label from discovered targets, and copies specified labels from the matched target to the profile.

You can specify multiple pyroscope.enrich components by giving them different labels.

Usage

pyroscope.enrich "<LABEL>" {
  targets = <DISCOVERY_COMPONENT>.targets
  target_match_label = "<LABEL>"
  forward_to = [<RECEIVER_LIST>]
}

Arguments

You can use the following arguments with pyroscope.enrich:

Name	Type	Description	Required
`forward_to`	`list(ProfilesReceiver)`	List of receivers to send enriched profiles to.	yes
`target_match_label`	`string`	The label from discovered targets to match against.	yes
`targets`	`list(map(string))`	List of targets from a discovery component.	yes
`labels_to_copy`	`list(string)`	List of labels to copy from discovered targets to profiles. If empty, `pyroscope.enrich` copies all labels.	no
`profiles_match_label`	`string`	The label from incoming profiles to match against discovered targets.	no

If profiles_match_label isn’t provided, the component uses target_match_label for matching profile labels.

Blocks

pyroscope.enrich doesn’t support any blocks. You can configure this component with arguments.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`receiver`	`ProfilesReceiver`	The receiver for profiles.

Component health

pyroscope.enrich is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

pyroscope.enrich doesn’t expose any component-specific debug information.

Debug metrics

pyroscope.enrich doesn’t expose any component-specific debug metrics.

Example

This example enriches profiles received over HTTP with metadata from Kubernetes service discovery:

// Discover Kubernetes pods
discovery.kubernetes "pods" {
  role = "pod"
}

// Add custom labels from Kubernetes metadata
discovery.relabel "pods" {
  targets = discovery.kubernetes.pods.targets

  rule {
    source_labels = ["__meta_kubernetes_namespace"]
    target_label  = "namespace"
  }

  rule {
    source_labels = ["__meta_kubernetes_pod_node_name"]
    target_label  = "node"
  }

  rule {
    source_labels = ["__meta_kubernetes_pod_label_app"]
    target_label  = "app"
  }

  rule {
    source_labels = ["__meta_kubernetes_pod_label_environment"]
    target_label  = "environment"
  }

  rule {
    source_labels = ["__meta_kubernetes_pod_ip"]
    target_label  = "pod_ip"
  }
}

// Receive profiles over HTTP
pyroscope.receive_http "default" {
  http {
    listen_address = "0.0.0.0"
    listen_port    = 4040
  }
  forward_to = [pyroscope.enrich.metadata.receiver]
}

// Enrich profiles with Kubernetes metadata
pyroscope.enrich "metadata" {
  targets               = discovery.relabel.pods.output
  target_match_label    = "pod_ip"
  profiles_match_label  = "service_name"
  labels_to_copy        = ["namespace", "node", "app", "environment"]
  forward_to            = [pyroscope.write.default.receiver]
}

// Write profiles to Pyroscope
pyroscope.write "default" {
  endpoint {
    url = "http://pyroscope:4040"
  }
}

Component behavior

The component matches profiles to discovered targets and enriches them with additional labels:

For each profile, it looks up the value of profiles_match_label from the profile’s labels, or target_match_label if profiles_match_label isn’t specified.
It matches this value against the target_match_label in discovered targets.
When it finds a match, it copies the requested labels_to_copy from the discovered target to the profile. If labels_to_copy is empty, it copies all labels.
The component forwards the profile, enriched or unchanged, to the configured receivers.

Caution
By default, pyroscope.enrich is ready as it starts, even if discovery doesn’t find targets. If you send profiles to this component before the metadata synchronizes, the component passes them through as-is, without enrichment. This is most likely to impact pyroscope.enrich on startup for a short time before discovery components send a list of targets.

Compatible components

pyroscope.enrich can accept arguments from the following components:

Components that export Targets
Components that export Pyroscope ProfilesReceiver

pyroscope.enrich has exports that can be consumed by the following components:

Components that consume Pyroscope ProfilesReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

pyroscope.java

Wed, 07 Oct 2026 16:23:35 +0000

`pyroscope.java`

pyroscope.java continuously profiles Java processes running on the local Linux OS using async-profiler.

Note
To use the pyroscope.java component you must run Alloy as root and inside host PID namespace.

You can specify multiple pyroscope.java components by giving them different labels.

Usage

pyroscope.java "<LABEL>" {
  targets    = <TARGET_LIST>
  forward_to = <RECEIVER_LIST>
}

Target JVM configuration

When you use pyroscope.java to profile Java applications, you can configure the target JVMs with some command line flags that ensure accurate profiling, especially for inlined methods. Add the following flags to your Java application’s startup command:

-XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints

For more details, refer to Restrictions/Limitations in the async-profiler documentation.

Additional configuration for Linux capabilities

If your Kubernetes environment has Linux capabilities enabled, configure the following in your Helm values to ensure pyroscope.java functions properly:

alloy:
  securityContext:
    runAsUser: 0
    runAsNonRoot: false
    capabilities:
      add:
        - PERFMON
        - SYS_PTRACE
        - SYS_RESOURCE
        - SYS_ADMIN

These capabilities enable Alloy to access performance monitoring subsystems, trace processes, override resource limits, and perform necessary system administration tasks for profiling.

Note
Adjust capabilities based on your specific security requirements and environment, following the principle of least privilege. The capability behavior depends on Container Runtime Interface (CRI) settings. For example, Docker drops capabilities that aren’t on the allowlist by default.

Arguments

You can use the following arguments with pyroscope.java:

Name	Type	Description	Default	Required
`forward_to`	`list(ProfilesReceiver)`	List of receivers to send collected profiles to.		yes
`targets`	`list(map(string))`	List of java process targets to profile.		yes
`tmp_dir`	`string`	Temporary directory to store async-profiler.	`"/tmp"`	no

After component startup, pyroscope.java creates a temporary directory under the tmp_dir argument and extracts the async-profiler binaries for both glibc and musl into the directory with the following layout.

/tmp/alloy-asprof-glibc-{SHA1}/bin/asprof
/tmp/alloy-asprof-glibc-{SHA1}/lib/libasyncProfiler.so
/tmp/alloy-asprof-musl-{SHA1}/bin/asprof
/tmp/alloy-asprof-musl-{SHA1}/lib/libasyncProfiler.so

After process profiling startup, the component detects libc type and copies the matching libasyncProfiler.so into the target process’s file system at the exact same path.

Note
The asprof binary runs with root permissions. If you change the tmp_dir argument to something other than /tmp, then you must ensure that the directory is only writable by root.

The filesystem mounted at tmp_dir in the Alloy and target containers must allow file execution. A mount option such as noexec prevents this.

Each target in the targets argument must always include the special __process_pid__ label, which corresponds to the process PID that’s used for profiling.

Warning
pyroscope.java doesn’t fail or report unhealthy if a target is missing the __process_pid__ label. It logs an error and silently skips profiling that target instead.

The component treats labels that start with a double underscore as internal and removes them before scraping.

Every profile carries a service_name label. You can set it on the target yourself. If you don’t, pyroscope.java adds the label and infers a value from the following sources, in this order:

__meta_kubernetes_pod_annotation_pyroscope_io_service_name which is a pyroscope.io/service_name Pod annotation.
__meta_kubernetes_namespace and __meta_kubernetes_pod_container_name, combined into java/<namespace>/<container_name>.
__meta_docker_container_name
__meta_dockerswarm_container_label_service_name or __meta_dockerswarm_service_name

When you don’t set service_name and the component can’t infer a value, it uses unspecified.

Injected labels

Independently of the targets argument, pyroscope.java automatically injects the following labels into collected profiles if you haven’t already defined them:

Label	Description
`otel.scope.name`	The instrumentation scope name, set to `com.grafana.alloy/pyroscope.java`.
`otel.scope.version`	The Alloy version that produced the profile.

Alloy only sets otel.scope.name and otel.scope.version if they aren’t already present on the profile. otel.scope.version is only set when otel.scope.name matches the default value.

Blocks

You can use the following block with pyroscope.java:

Block	Description	Required
`profiling_config`	Describes java profiling configuration.	no

`profiling_config`

The profiling_config block describes how the component invokes async-profiler.

You can use the following arguments with the profiling_config block:

Name	Type	Description	Default	Required
`alloc`	`string`	Allocation profiling sampling configuration. It’s passed as an `--alloc` argument to async-profiler.	`"512k"`	no
`cpu`	`bool`	A flag to enable CPU profiling, using `itimer` async-profiler event by default.	`true`	no
`custom_arguments`	`list(string)`	Sends raw args to async-profiler, skipping all Alloy arguments except `interval`. Refer to `custom_arguments`.	`[]`	no
`event`	`string`	Sets the CPU profiling event. Refer to `event` for supported values.	`"itimer"`	no
`interval`	`duration`	How frequently to collect profiles from the targets.	`"60s"`	no
`lock`	`string`	Lock profiling sampling configuration. It’s passed as an `--lock` argument to async-profiler.	`"10ms"`	no
`log_level`	`string`	Sets the log level in async profiler. One of `TRACE`, `DEBUG`, `INFO`, `WARN`, `ERROR`, or `NONE`.	`"INFO"`	no
`per_thread`	`bool`	Sets per thread mode on async profiler. It’s passed as an `-t` argument to async-profiler.	`false`	no
`quiet`	`bool`	If set, suppresses the `Profiling started/stopped` log message.	`false`	no
`sample_rate`	`int`	CPU profiling sample rate. It’s converted from Hz to interval and passed as an `-i` argument to async-profiler.	`100`	no
`tlab`	`bool`	Enables TLAB-based allocation events. Passed as `--tlab` to async-profiler. Requires `alloc`.	`false`	no

Refer to profiler-options for more information about async-profiler configuration.

`custom_arguments`

custom_arguments passes async-profiler start flags directly.

When you set custom_arguments, Alloy skips these options from this block:

cpu, event, per_thread, sample_rate, alloc, lock, log_level, tlab.

For example, this configuration enables multi-event profiling for cpu, alloc, and lock with custom thresholds:

pyroscope.java "java" {
  targets    = discovery.relabel.java.output
  forward_to = [pyroscope.write.staging.receiver]

  profiling_config {
    interval = "60s"
    custom_arguments = ["-e", "cpu,alloc,lock", "--alloc", "2m", "--lock", "10ms"]
  }
}

Refer to Profiling modes and profiler-options for the complete async-profiler option list.

`event`

The event argument configures the profiling mode used by async-profiler. async-profiler supports various profiling modes including CPU profiling, wall-clock profiling, and hardware performance monitoring events. For a complete overview of all available profiling modes and their use cases, refer to Profiling modes in the async-profiler documentation.

`per_thread`

Warning
The per_thread option doesn’t apply when using JFR output format. Since pyroscope.java uses JFR format exclusively, this option has no effect. For more details, refer to Options applicable to any output format except JFR in the async-profiler documentation.

Exported fields

pyroscope.java doesn’t export any fields.

Component health

pyroscope.java is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

pyroscope.java doesn’t expose any component-specific debug information.

Debug metrics

pyroscope.java doesn’t expose any component-specific debug metrics.

Example

The following example profiles every Java process on the current host:

pyroscope.write "staging" {
  endpoint {
    url = "http://localhost:4040"
  }
}

discovery.process "all" {
  refresh_interval = "60s"
  discover_config {
    cwd = true
    exe = true
    commandline = true
    username = true
    uid = true
    container_id = true
  }
}

discovery.relabel "java" {
  targets = discovery.process.all.targets
  rule {
    action = "keep"
    regex = ".*/java$"
    source_labels = ["__meta_process_exe"]
  }
}

pyroscope.java "java" {
  targets = discovery.relabel.java.output
  forward_to = [pyroscope.write.staging.receiver]
  profiling_config {
    interval = "60s"
    alloc = "512k"
    cpu = true
    sample_rate = 100
    lock = "1ms"
  }
}

Compatible components

pyroscope.java can accept arguments from the following components:

Components that export Targets
Components that export Pyroscope ProfilesReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

pyroscope.receive_http

Wed, 07 Oct 2026 16:23:35 +0000

`pyroscope.receive_http`

pyroscope.receive_http receives profiles over HTTP and forwards them to pyroscope.* components capable of receiving profiles.

The HTTP API exposed is compatible with both the Pyroscope HTTP ingest API and the pushv1.PusherService Connect API. This allows pyroscope.receive_http to act as a proxy for Pyroscope profiles, enabling flexible routing and distribution of profile data.

You can specify multiple pyroscope.receive_http components by giving them different labels.

Usage

pyroscope.receive_http "<LABEL>" {
  http {
    listen_address = "<LISTEN_ADDRESS>"
    listen_port = "<PORT>"
  }
  forward_to = <RECEIVER_LIST>
}

The component starts an HTTP server supporting the following endpoints:

POST /ingest: Send profiles to the component, which forwards them to the receivers configured in the forward_to argument. The request format must match the format of the Pyroscope ingest API.
POST /push.v1.PusherService/Push: Send profiles to the component, which forwards them to the receivers configured in the forward_to argument. The request format must match the format of the Pyroscope pushv1.PusherService Connect API.

Arguments

You can use the following argument with pyroscope.receive_http:

Name	Type	Description	Default	Required
`forward_to`	`list(ProfilesReceiver)`	List of receivers to send profiles to.		yes

Blocks

You can use the following blocks with pyroscope.receive_http:

Block	Description	Required
`profiling_config`	Describes java profiling configuration.	no

`http`

The http block configures the HTTP server.

You can use the following arguments to configure the http block. Any omitted fields take their default values.

Name	Type	Description	Default	Required
`conn_limit`	`int`	Maximum number of simultaneous HTTP connections. Defaults to no limit.	`0`	no
`listen_address`	`string`	Network address on which the server listens for new connections. Defaults to accepting all incoming connections.	`""`	no
`listen_port`	`int`	Port number on which the server listens for new connections.	`8080`	no
`server_idle_timeout`	`duration`	Idle timeout for HTTP server.	`"120s"`	no
`server_read_timeout`	`duration`	Read timeout for HTTP server.	`"30s"`	no
`server_write_timeout`	`duration`	Write timeout for HTTP server.	`"30s"`	no

`tls`

The tls block configures TLS for the HTTP server.

Name	Type	Description	Default	Required
`cert_pem`	`string`	PEM data of the server TLS certificate.	`""`	no
`cert_file`	`string`	Path to the server TLS certificate on disk.	`""`	no
`client_auth_type`	`string`	Client authentication to use.	`"NoClientCert"`	no
`client_ca_file`	`string`	Path to the client CA file on disk to validate requests against.	`""`	no
`client_ca_pem`	`string`	PEM data of the client CA to validate requests against.	`""`	no
`key_file`	`string`	Path to the server TLS key on disk.	`""`	no
`key_pem`	`secret`	PEM data of the server TLS key.	`""`	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

Exported fields

pyroscope.receive_http doesn’t export any fields.

Component health

pyroscope.receive_http is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

pyroscope.receive_http doesn’t expose any component-specific debug information.

Debug metrics

The following Prometheus metrics are exposed:

Name	Type	Description
`pyroscope_receive_http_tcp_connections`	`gauge`	Current number of accepted TCP connections.
`pyroscope_receive_http_tcp_connections_limit`	`gauge`	The maximum number of TCP connections that the component can accept. A value of `0` means no limit.

Troubleshoot

Connection limit errors

When using pyroscope.write to push profiles to a pyroscope.receive_http component, you may encounter errors like:

"failed to push to endpoint" err="deadline_exceeded: context deadline exceeded"

This typically indicates that the receiving component has reached its TCP connection limit.

To resolve connection limit errors, first diagnose the issue, then apply one of the solutions.

Diagnose connection limit issues

Check the connection metrics on the pyroscope.receive_http component:
- pyroscope_receive_http_tcp_connections: Current number of accepted TCP connections
- pyroscope_receive_http_tcp_connections_limit: Maximum number of TCP connections allowed
If the current connections are approaching or at the limit, you need to take action.

Solution 1: Increase the connection limit

To increase the connection limit, increase the conn_limit parameter in the pyroscope.receive_http configuration:

pyroscope.receive_http "example" {
  http {
    conn_limit = 32768  // Increase from default 16384
    // ... other settings
  }
  // ... rest of configuration
}

Solution 2: Horizontal scaling

To distribute the connection load across multiple receivers, deploy multiple instances of pyroscope.receive_http behind a load balancer.

Timeout chain issues

When chaining multiple Pyroscope components such as pyroscope.write to pyroscope.receive_http to another pyroscope.write, you may encounter timeout issues that prevent retries:

"failed to push to endpoint" err="deadline_exceeded: context deadline exceeded"

Understand the problem

This issue occurs when:

The first pyroscope.write component sends profiles to pyroscope.receive_http.
The pyroscope.receive_http component forwards profiles to another pyroscope.write component.
Both pyroscope.write components have the same default remote_timeout of 10 seconds.
The request context passes from the first pyroscope.write through pyroscope.receive_http to the second pyroscope.write, maintaining the original 10-second deadline.
If the second pyroscope.write component’s downstream request takes the full 10 seconds due to a broken TCP idle connection, there’s no time left for retries.

To resolve timeout chain issues, apply one of the following solutions.

Solution 1: Increase timeout on the first pyroscope.write

To provide buffer time for retries, increase the remote_timeout on the initial pyroscope.write component:

pyroscope.write "w1" {
  endpoint {
    url = "http://pyroscope-receiver:8080"
    remote_timeout = "30s"  // Increased from default 10s
    // ... other settings
  }
  // ... rest of configuration
}

Solution 2: Decrease timeout on the downstream pyroscope.write

To ensure faster failures and allow time for retries, reduce the remote_timeout on the downstream pyroscope.write component:

pyroscope.write "w2" {
  endpoint {
    url = "http://pyroscope-backend:4040"
    remote_timeout = "3s"  // Reduced from default 10s
    // ... other settings
  }
  // ... rest of configuration
}

Important considerations

Normal latency: Pyroscope servers with the new architecture typically have 500-1000ms average latency for requests
Timeout buffer: Always leave sufficient buffer time for retries when chaining components
Retry configuration: Consider adjusting max_backoff_retries and backoff periods alongside timeout values
Monitoring: Monitor the pyroscope_write_latency metric to understand actual request latencies and adjust timeouts accordingly

Example

This example creates a pyroscope.receive_http component, which starts an HTTP server listening on 0.0.0.0 and port 9999. The server receives profiles and forwards them to multiple pyroscope.write components, which write these profiles to different HTTP endpoints.

// Receives profiles over HTTP
pyroscope.receive_http "default" {
  http {
    listen_address = "0.0.0.0"
    listen_port = 9999
  }
  forward_to = [pyroscope.write.staging.receiver, pyroscope.write.production.receiver]
}

// Send profiles to a staging Pyroscope instance
pyroscope.write "staging" {
  endpoint {
    url = "http://pyroscope-staging:4040"
  }
}

// Send profiles to a production Pyroscope instance
pyroscope.write "production" {
  endpoint {
    url = "http://pyroscope-production:4040"
  }
}

Note
This example demonstrates forwarding to multiple pyroscope.write components. This configuration duplicates the received profiles and sends a copy to each configured pyroscope.write component.

You can also create multiple pyroscope.receive_http components with different configurations to listen on different addresses or ports as needed. This flexibility allows you to design a setup that best fits your infrastructure and profile routing requirements.

Compatible components

pyroscope.receive_http can accept arguments from the following components:

Components that export Pyroscope ProfilesReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

pyroscope.relabel

Wed, 07 Oct 2026 16:23:35 +0000

`pyroscope.relabel`

The pyroscope.relabel component rewrites the external label set of each profile passed to its receiver by applying one or more relabeling rules and forwards the results to the list of receivers.

When no rules apply to a profile, the component forwards that profile as-is to each receiver in its arguments. The component drops the profile when no external labels remain after it applies the rules.

pyroscope.relabel only rewrites labels that aren’t embedded in the profile itself, such as labels inferred by pyroscope.scrape or labels provided through the /ingest?name=... query parameter. It doesn’t parse or modify labels embedded inside profile payloads like pprof sample labels.

The most common use of pyroscope.relabel is to filter profiles or standardize external labels passed to one or more downstream receivers. The component applies each rule block to the label set of every profile in the order the blocks appear in your configuration file.

You can specify multiple pyroscope.relabel components by giving them different labels.

Usage

pyroscope.relabel "<LABEL>" {
    forward_to = <RECEIVER_LIST>

    rule {
        ...
    }

    ...
}

Arguments

You can use the following arguments with pyroscope.relabel:

Name	Type	Description	Default	Required
`forward_to`	`list(ProfilesReceiver)`	List of receivers to forward profiles to after relabeling		yes
`max_cache_size`	`number`	Maximum number of entries in the label cache	`10000`	no

Blocks

You can use the following block with pyroscope.relabel:

Block	Description	Required
`profiling_config`	Describes java profiling configuration.	no

`rule`

The rule block contains the definition of any relabeling rules that can be applied to an input metric. If more than one rule block is defined, the transformations are applied in top-down order.

The following arguments can be used to configure a rule. All arguments are optional. Omitted fields take their default values.

Name	Type	Description	Default	Required
`action`	`string`	The relabeling action to perform.	replace	no
`modulus`	`uint`	A positive integer used to calculate the modulus of the hashed source label values.		no
`regex`	`string`	A valid RE2 expression with support for parenthesized capture groups. Used to match the extracted value from the combination of the `source_label` and `separator` fields or filter labels during the `labelkeep/labeldrop/labelmap` actions.	`(.*)`	no
`replacement`	`string`	The value against which a regular expression replace is performed, if the regular expression matches the extracted value. Supports previously captured groups.	`"$1"`	no
`separator`	`string`	The separator used to concatenate the values present in `source_labels`.	;	no
`source_labels`	`list(string)`	The list of labels whose values are to be selected. Their content is concatenated using the `separator` and matched against `regex`.		no
`target_label`	`string`	Label to which the resulting value will be written to.		no

You can use the following actions:

drop - Drops metrics where regex matches the string extracted using the source_labels and separator.
dropequal - Drop targets for which the concatenated source_labels do match target_label.
hashmod - Hashes the concatenated labels, calculates its modulo modulus and writes the result to the target_label.
keep - Keeps metrics where regex matches the string extracted using the source_labels and separator.
keepequal - Drop targets for which the concatenated source_labels don’t match target_label.
labeldrop - Matches regex against all label names. Any labels that match are removed from the metric’s label set.
labelkeep - Matches regex against all label names. Any labels that don’t match are removed from the metric’s label set.
labelmap - Matches regex against all label names. Any labels that match are renamed according to the contents of the replacement field.
lowercase - Sets target_label to the lowercase form of the concatenated source_labels.
replace - Matches regex to the concatenated labels. If there’s a match, it replaces the content of the target_label using the contents of the replacement field.
uppercase - Sets target_label to the uppercase form of the concatenated source_labels.

Note
The regular expression capture groups can be referred to using either the $CAPTURE_GROUP_NUMBER or ${CAPTURE_GROUP_NUMBER} notation.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`receiver`	`ProfilesReceiver`	A receiver that accepts profiles for relabeling.
`rules`	`RelabelRules`	The list of relabeling rules.

Component health

pyroscope.relabel is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

pyroscope.relabel doesn’t expose any component-specific debug information.

Debug metrics

The following Prometheus metrics are exposed:

Name	Type	Description
`pyroscope_relabel_cache_hits`	`counter`	Total number of cache hits.
`pyroscope_relabel_cache_misses`	`counter`	Total number of cache misses.
`pyroscope_relabel_cache_size`	`gauge`	Total size of relabel cache.
`pyroscope_relabel_profiles_dropped`	`counter`	Total number of profiles dropped by relabeling rules.
`pyroscope_relabel_profiles_processed`	`counter`	Total number of profiles processed.
`pyroscope_relabel_profiles_written`	`counter`	Total number of profiles forwarded.
`pyroscope_fanout_latency`	`histogram`	Write latency for sending to direct and indirect components.
`pyroscope_forwarded_entries_total`	`counter`	Total number of samples sent to downstream components.

Example

pyroscope.receive_http "default" {
    forward_to = [pyroscope.relabel.filter_profiles.receiver]

    http {
        listen_address = "0.0.0.0"
        listen_port = 9999
    }
}

pyroscope.relabel "filter_profiles" {
    forward_to = [pyroscope.write.staging.receiver]

    // This creates a consistent hash value (0 or 1) for each unique combination of labels
    // Using multiple source labels provides better sampling distribution across your profiles
    rule {
        source_labels = ["env"]
        target_label = "__tmp_hash"
        action = "hashmod"
        modulus = 2
    }

    // This effectively samples ~50% of profile series
    // The same combination of source label values will always hash to the same number,
    // ensuring consistent sampling
    rule {
        source_labels = ["__tmp_hash"]
        action       = "drop"
        regex        = "^1$"
    }
}

pyroscope.write "staging" {
  endpoint {
    url = "http://pyroscope-staging:4040"
  }
}

Compatible components

pyroscope.relabel can accept arguments from the following components:

Components that export Pyroscope ProfilesReceiver

pyroscope.relabel has exports that can be consumed by the following components:

Components that consume Pyroscope ProfilesReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

pyroscope.scrape

Wed, 07 Oct 2026 16:23:35 +0000

`pyroscope.scrape`

pyroscope.scrape collects pprof performance profiles for a given set of HTTP targets.

pyroscope.scrape mimics the scraping behavior of prometheus.scrape. Similarly to how Prometheus scrapes metrics via HTTP, pyroscope.scrape collects profiles via HTTP requests.

Unlike Prometheus, which usually only scrapes one /metrics endpoint per target, pyroscope.scrape may need to scrape multiple endpoints for the same target. This is because the component scrapes different profile types from different endpoints. For example, the component may scrape mutex profiles from a /debug/pprof/delta_mutex HTTP endpoint, and memory consumption from a /debug/pprof/allocs HTTP endpoint.

You can configure the profile paths, protocol scheme, scrape interval, scrape timeout, query parameters, and any other settings within pyroscope.scrape.

The pyroscope.scrape component regards a scrape as successful if it responded with an HTTP 200 OK status code and returned the body of a valid pprof profile.

If a scrape request fails, the debug UI for pyroscope.scrape shows:

Detailed information about the failure.
The time of the last successful scrape.
The labels last used for scraping.

The component can forward the scraped performance profiles to components such as pyroscope.write through the forward_to argument.

You can specify multiple pyroscope.scrape components by giving them different labels.

Usage

pyroscope.scrape "<LABEL>" {
  targets    = <TARGET_LIST>
  forward_to = <RECEIVER_LIST>
}

Arguments

pyroscope.scrape starts a scrape job to scrape all of the input targets. The component can start multiple scrape jobs for a single input target when it scrapes multiple profile types.

You can use the following arguments with pyroscope.scrape:

Name	Type	Description	Default	Required
`forward_to`	`list(ProfilesReceiver)`	List of receivers to send scraped profiles to.		yes
`targets`	`list(map(string))`	List of targets to scrape.		yes
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`delta_profiling_duration`	`duration`	How long to scrape a delta profile. Must be larger than 1 second.	`"14s"`	no
`enable_http2`	`bool`	Whether to support HTTP2 for requests.	`true`	no
`follow_redirects`	`bool`	Whether to follow redirects from the server.	`true`	no
`http_headers`	`map(list(secret))`	Custom HTTP headers to send with each request. The map key is the header name.		no
`job_name`	`string`	The job name to override the job label with.	component name	no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`params`	`map(list(string))`	A set of query parameters to send when scraping the target.		no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`scheme`	`string`	The URL scheme with which to fetch profiles from targets.	`"http"`	no
`scrape_interval`	`duration`	How frequently to scrape the targets of this scrape configuration.	`"15s"`	no
`scrape_timeout`	`duration`	The timeout for scraping targets of this configuration.	`"10s"`	no

You can provide at most one of the following:

Any omitted arguments take on their default values. If you pass conflicting arguments, for example, configuring both bearer_token and bearer_token_file, then pyroscope.scrape fails to start and reports an error.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_from_environment uses the environment variables HTTP_PROXY, HTTPS_PROXY, and NO_PROXY (or the lowercase versions thereof). Requests use the proxy from the environment variable matching their scheme, unless excluded by NO_PROXY. proxy_url and no_proxy must not be configured if proxy_from_environment is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

`job_name`

The job_name argument defaults to the component’s unique identifier.

For example, the job_name of pyroscope.scrape "local" { ... } is "pyroscope.scrape.local".

`targets`

You can define the list of targets statically, dynamically, or as a combination of both.

The following special labels can change the behavior of pyroscope.scrape:

__address__ is the special label that must always be present and corresponds to the <host>:<port> that’s used for the scrape request.
__name__ is the special label that indicates which profile type the component collects.
__profile_path__ is the special label that holds the path to the profile endpoint on the target (for example, “/debug/pprof/allocs”).
__profile_path_prefix__ is the special label that holds an optional prefix to prepend to the profile path (for example, "/mimir-prometheus").
service_name is a required label that identifies the service you profile.

The component treats labels that start with a double underscore as internal and removes them before scraping.

Every profile carries a service_name label. You can set it on the target yourself. If you don’t, pyroscope.scrape adds the label and infers a value from the following sources, in this order:

__meta_kubernetes_pod_annotation_pyroscope_io_service_name which is a pyroscope.io/service_name Pod annotation.
__meta_kubernetes_namespace and __meta_kubernetes_pod_container_name
__meta_docker_container_name
__meta_dockerswarm_container_label_service_name or __meta_dockerswarm_service_name

When you don’t set service_name and the component can’t infer a value, it uses unspecified.

The component injects the following labels into the scraped profiles so that you can link them to a scrape target:

Label	Description
`"job"`	The `job_name` that the target belongs to.
`"instance"`	The `__address__` or `<host>:<port>` of the scrape target’s URL.
`"service_name"`	The inferred Pyroscope service name.
`"otel.scope.name"`	The instrumentation scope name, set to `com.grafana.alloy/pyroscope.scrape`.
`"otel.scope.version"`	The Alloy version that produced the profile.

Alloy only sets otel.scope.name and otel.scope.version if they aren’t already present on the scraped profile. otel.scope.version is only set when otel.scope.name matches the default value.

`scrape_interval`

The scrape_interval typically refers to the frequency with which Alloy collects performance profiles from the monitored targets. It represents the time interval between consecutive scrapes or data collection events. This parameter is important for controlling the trade-off between resource usage and the freshness of the collected data.

If scrape_interval is short:

Advantages:
- The component loses fewer profiles if the scraped application crashes.
Disadvantages:
- Greater consumption of CPU, memory, and network resources during scrapes and remote writes.
- The backend database (Pyroscope) consumes more storage space.

If scrape_interval is long:

Advantages:
- Lower resource consumption.
Disadvantages:
- The component loses more profiles if the scraped application crashes.
- If you set the delta argument to true, the batch size of each remote write to Pyroscope may be bigger. You may need to tune the Pyroscope database with higher limits.
- If you set the delta argument to true, you run a larger risk of reaching the HTTP server timeouts of the scraped application.

For example, consider this situation:

You configure pyroscope.scrape with a scrape_interval of "60s".
The scraped application runs an HTTP server with a timeout of 30 seconds.
Any scrape HTTP requests where you set the delta argument to true fail, because they attempt to run for 59 seconds.

Blocks

You can use the following blocks with pyroscope.scrape:

Block	Description	Required
`profiling_config`	Describes java profiling configuration.	no

Any omitted blocks take on their default values. For example, if you don’t specify profile.mutex in the configuration, the component uses the defaults documented in profile.mutex.

`authorization`

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

Warning
Using credentials_file causes the file to be read on every outgoing request. Use the local.file component with the credentials attribute instead to avoid unnecessary reads.

`basic_auth`

Name	Type	Description	Required
`password_file`	`string`	Path to a file that contains the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

When you use password_file, the file is read on every outgoing request that uses basic authentication.

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

Note
High scrape or write rates create repeated file reads when you use password_file. You can use the local.file component to read the password file and provide the content to the password attribute. This avoids repeated file reads because local.file monitors the file and reads when it changes.

`clustering`

Name	Type	Description	Default	Required
`enabled`	`bool`	Enables sharing targets with other cluster nodes.	`false`	yes

When Alloy is using clustering, and you set enabled to true, this pyroscope.scrape component instance opts in to participating in the cluster to distribute scrape load between all cluster nodes.

Clustering causes the set of targets to be locally filtered down to a unique subset per node, where each node is roughly assigned the same number of targets. If the state of the cluster changes, such as when a node joins, the cluster recalculates the subset of targets to scrape per node.

When you enable clustering mode, all Alloy instances participating in the cluster must use the same configuration file and have access to the same service discovery APIs.

If Alloy is not running in clustered mode, this block is a no-op.

`oauth2`

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

Warning
Using client_secret_file causes the file to be read on every outgoing request. Use the local.file component with the client_secret attribute instead to avoid unnecessary reads.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

`profiling_config`

The profiling_config block configures the profiling settings when scraping targets.

You can use the following arguments with the profiling_config block:

Name	Type	Description	Default	Required
`path_prefix`	`string`	The path prefix to use when scraping targets.		no

`profile.block`

The profile.block block collects profiles on process blocking.

You can use the following arguments with the profile.block block:

Name	Type	Description	Default	Required
`delta`	`boolean`	Whether to scrape the profile as a delta.	`false`	no
`enabled`	`boolean`	Enable scraping for this profile type.	`true`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/pprof/block"`	no

For more information about the delta argument, see the delta argument section.

`profile.custom`

The profile.custom block collects profiles from custom endpoints. You must give each block a label:

profile.custom "<PROFILE_TYPE>" {
  enabled = true
  path    = "<PROFILE_PATH>"
}

You can specify multiple profile.custom blocks. Labels assigned to profile.custom blocks must be unique across the component.

You can use the following arguments with the profile.custom block:

Name	Type	Description	Default	Required
`delta`	`boolean`	Whether to scrape the profile as a delta.	`false`	no
`enabled`	`boolean`	Enable scraping for this profile type.		yes
`path`	`string`	The path to the profile type on the target.		yes

When the delta argument is true, a seconds query parameter is automatically added to requests. The seconds used is equal to scrape_interval - 1.

`profile.fgprof`

The profile.fgprof block collects profiles from an fgprof endpoint.

You can use the following arguments with the profile.fgprof block:

Name	Type	Description	Default	Required
`delta`	`boolean`	Whether to scrape the profile as a delta.	`true`	no
`enabled`	`boolean`	Enable scraping for this profile type.	`false`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/fgprof"`	no

For more information about the delta argument, see the delta argument section.

`profile.godeltaprof_block`

The profile.godeltaprof_block block collects profiles from the godeltaprof block endpoint. The target computes the delta.

You can use the following arguments with the profile.godeltaprof_block block:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enable scraping for this profile type.	`false`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/pprof/delta_block"`	no

`profile.godeltaprof_memory`

The profile.godeltaprof_memory block collects profiles from the godeltaprof memory endpoint. The target computes the delta.

You can use the following arguments with the profile.godeltaprof_memory block:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enable scraping for this profile type.	`false`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/pprof/delta_heap"`	no

`profile.godeltaprof_mutex`

The profile.godeltaprof_mutex block collects profiles from the godeltaprof mutex endpoint. The target computes the delta.

You can use the following arguments with the profile.godeltaprof_mutex block:

Name	Type	Description	Default	Required
`enabled`	`boolean`	Enable scraping for this profile type.	`false`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/pprof/delta_mutex"`	no

`profile.goroutine`

The profile.goroutine block collects profiles on the number of goroutines.

You can use the following arguments with the profile.goroutine block:

Name	Type	Description	Default	Required
`delta`	`boolean`	Whether to scrape the profile as a delta.	`false`	no
`enabled`	`boolean`	Enable scraping for this profile type.	`true`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/pprof/goroutine"`	no

Refer to delta argument for more information about the delta argument.

`profile.memory`

The profile.memory block collects profiles on memory consumption.

You can use the following arguments with the profile.memory block:

Name	Type	Description	Default	Required
`delta`	`boolean`	Whether to scrape the profile as a delta.	`false`	no
`enabled`	`boolean`	Enable scraping for this profile type.	`true`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/pprof/allocs"`	no

Refer to delta argument for more information about the delta argument.

`profile.mutex`

The profile.mutex block collects profiles on mutexes.

You can use the following arguments with the profile.mutex block:

Name	Type	Description	Default	Required
`delta`	`boolean`	Whether to scrape the profile as a delta.	`false`	no
`enabled`	`boolean`	Enable scraping for this profile type.	`true`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/pprof/mutex"`	no

Refer to delta argument for more information about the delta argument.

`profile.process_cpu`

The profile.process_cpu block collects profiles on CPU consumption for the process.

You can use the following arguments with the profile.process_cpu block:

Name	Type	Description	Default	Required
`delta`	`boolean`	Whether to scrape the profile as a delta.	`true`	no
`enabled`	`boolean`	Enable scraping for this profile type.	`true`	no
`path`	`string`	The path to the profile type on the target.	`"/debug/pprof/profile"`	no

For more information about the delta argument, see the delta argument section.

`tls_config`

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

When min_version isn’t provided, the minimum acceptable TLS version is inherited from Go’s default minimum version, TLS 1.2. If min_version is provided, it must be set to one of the following strings:

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Caution
If you set insecure_skip_verify to true, you disable verification of the server’s certificate chain and hostname. Alloy accepts any certificate the server presents, including expired, self-signed, or invalid certificates.

You can use insecure_skip_verify for local development, self-signed certificates, or endpoints that use a private CA outside the system trust store. If the hostname doesn’t match the certificate, set server_name instead of insecure_skip_verify. In production, use ca_file or ca_pem to trust a private CA instead of insecure_skip_verify.

Set insecure_skip_verify to true only in isolated development or testing environments where you don’t transmit sensitive data. Remove the setting before you deploy to production.

Common configuration

The following configuration applies across the profile types described in Blocks.

`delta` argument

When the delta argument is false, the pprof HTTP query is instantaneous.

When the delta argument is true:

The pprof HTTP query runs for a certain amount of time.
A seconds parameter is automatically added to the HTTP request.
The default value for the seconds query parameter is scrape_interval - 1. If you set delta_profiling_duration, then seconds takes the same value as delta_profiling_duration. However, the delta_profiling_duration can’t be larger than scrape_interval. For example, if you set scrape_interval to "15s", then seconds defaults to 14s If you set delta_profiling_duration to 16s, then you must set scrape_interval to at least 17s. If the HTTP endpoint is /debug/pprof/profile, then the HTTP query becomes /debug/pprof/profile?seconds=14

Exported fields

pyroscope.scrape doesn’t export any fields.

Component health

pyroscope.scrape is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

pyroscope.scrape reports the status of the last scrape for each configured scrape job on the component’s debug endpoint.

Debug metrics

The following Prometheus metrics are exposed:

Name	Type	Description
`pyroscope_fanout_latency`	`histogram`	Write latency for sending to direct and indirect components.
`pyroscope_forwarded_entries_total`	`counter`	Total number of samples sent to downstream components.

Examples

The following examples show how to scrape profiles from static and dynamically discovered targets, and how to enable and disable specific profile types.

Default endpoints of static targets

The following example sets up a scrape job of a statically configured list of targets - Alloy itself and Pyroscope. The component sends the scraped profiles to pyroscope.write, which remote writes them to a Pyroscope database.

pyroscope.scrape "local" {
  targets = [
    {"__address__" = "localhost:4040", "service_name"="pyroscope"},
    {"__address__" = "localhost:12345", "service_name"="alloy"},
  ]

  forward_to = [pyroscope.write.local.receiver]
}

pyroscope.write "local" {
  endpoint {
    url = "http://pyroscope:4040"
  }
}

The component scrapes these endpoints every 15 seconds:

http://localhost:4040/debug/pprof/allocs
http://localhost:4040/debug/pprof/block
http://localhost:4040/debug/pprof/goroutine
http://localhost:4040/debug/pprof/mutex
http://localhost:4040/debug/pprof/profile?seconds=14

http://localhost:12345/debug/pprof/allocs
http://localhost:12345/debug/pprof/block
http://localhost:12345/debug/pprof/goroutine
http://localhost:12345/debug/pprof/mutex
http://localhost:12345/debug/pprof/profile?seconds=14

The component adds seconds=14 to the /debug/pprof/profile endpoint, because:

The delta argument of the profile.process_cpu block is true by default.
scrape_interval is "15s" by default.

The component doesn’t scrape the /debug/fgprof endpoint, because the enabled argument of the profile.fgprof block is false by default.

Default endpoints of dynamic targets

discovery.http "dynamic_targets" {
  url = "https://example.com/scrape_targets"
  refresh_interval = "15s"
}

pyroscope.scrape "local" {
  targets = [discovery.http.dynamic_targets.targets]

  forward_to = [pyroscope.write.local.receiver]
}

pyroscope.write "local" {
  endpoint {
    url = "http://pyroscope:4040"
  }
}

Default endpoints of static and dynamic targets

discovery.http "dynamic_targets" {
  url = "https://example.com/scrape_targets"
  refresh_interval = "15s"
}

pyroscope.scrape "local" {
  targets = array.concat([
    {"__address__" = "localhost:4040", "service_name"="pyroscope"},
    {"__address__" = "localhost:12345", "service_name"="alloy"},
  ], discovery.http.dynamic_targets.targets)

  forward_to = [pyroscope.write.local.receiver]
}

pyroscope.write "local" {
  endpoint {
    url = "http://pyroscope:4040"
  }
}

Enable and disable profiles

pyroscope.scrape "local" {
  targets = [
    {"__address__" = "localhost:12345", "service_name"="alloy"},
  ]

  profiling_config {
    profile.fgprof {
      enabled = true
    }
    profile.block {
      enabled = false
    }
    profile.mutex {
      enabled = false
    }
  }

  forward_to = [pyroscope.write.local.receiver]
}

The component scrapes these endpoints every 15 seconds:

http://localhost:12345/debug/pprof/allocs
http://localhost:12345/debug/pprof/goroutine
http://localhost:12345/debug/pprof/profile?seconds=14
http://localhost:12345/debug/fgprof?seconds=14

These endpoints are NOT scraped because they’re explicitly disabled:

http://localhost:12345/debug/pprof/block
http://localhost:12345/debug/pprof/mutex

Compatible components

pyroscope.scrape can accept arguments from the following components:

Components that export Targets
Components that export Pyroscope ProfilesReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.

pyroscope.write

Wed, 07 Oct 2026 16:23:35 +0000

`pyroscope.write`

pyroscope.write receives performance profiles from other components and forwards them to a series of user-supplied endpoints. When pyroscope.write forwards profiles, it drops all labels that start with a double underscore, with the following exceptions:

__name__, because it identifies the profile type.
__delta__, because delta profiles require it.

You can specify multiple pyroscope.write components by giving them different labels.

Usage

pyroscope.write "<LABEL>" {
  endpoint {
    url = "<PYROSCOPE_URL>"

    ...
  }

  ...
}

Arguments

You can use the following argument with pyroscope.write:

Name	Type	Description	Default	Required
`external_labels`	`map(string)`	Labels to add to profiles sent over the network.		no

Blocks

You can use the following blocks with pyroscope.write:

Block	Description	Required
`profiling_config`	Describes java profiling configuration.	no

`endpoint`

The endpoint block describes a single location to send profiles to. You can provide multiple endpoint blocks to send profiles to multiple locations.

You can use the following arguments with the endpoint block:

Name	Type	Description	Default	Required
`url`	`string`	Full URL to send profiles to.		yes
`bearer_token`	`secret`	Bearer token to authenticate with.		no
`bearer_token_file`	`string`	File containing a bearer token to authenticate with.		no
`enable_http2`	`bool`	Whether to support HTTP2 for requests.	`true`	no
`follow_redirects`	`bool`	Whether to follow redirects from the server.	`true`	no
`headers`	`map(string)`	Extra headers to deliver with the request.		no
`http_headers`	`map(list(secret))`	Custom HTTP headers to send with each request. The map key is the header name.		no
`max_backoff_period`	`duration`	Maximum backoff time between retries.	`"5m"`	no
`max_backoff_retries`	`int`	Maximum number of retries. 0 to retry infinitely.	`10`	no
`min_backoff_period`	`duration`	Initial backoff time between retries.	`"500ms"`	no
`name`	`string`	Optional name to identify the endpoint in metrics.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`remote_timeout`	`duration`	Timeout for requests made to the URL.	`"10s"`	no
`retry_on_http_429`	`bool`	Retry when the server returns an HTTP 429 status code.	`true`	no

You can provide at most one of the following:

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

When you provide multiple endpoint blocks, profiles are concurrently forwarded to all configured locations.

The retry_on_http_429 argument controls whether the component treats HTTP 429 responses as recoverable errors. Other HTTP 4xx status code responses are never considered recoverable errors, with the exception of HTTP 408 Request Timeout, which is always retried. When you enable retry_on_http_429, the min_backoff_period, max_backoff_period, and max_backoff_retries arguments control the retry backoff.

`authorization`

Name	Type	Description	Required
`credentials_file`	`string`	File containing the secret value.	no
`credentials`	`secret`	Secret value.	no
`type`	`string`	Authorization type, for example, “Bearer”.	no

credential and credentials_file are mutually exclusive, and only one can be provided inside an authorization block.

Warning
Using credentials_file causes the file to be read on every outgoing request. Use the local.file component with the credentials attribute instead to avoid unnecessary reads.

`basic_auth`

Name	Type	Description	Required
`password_file`	`string`	Path to a file that contains the basic auth password.	no
`password`	`secret`	Basic auth password.	no
`username`	`string`	Basic auth username.	no

When you use password_file, the file is read on every outgoing request that uses basic authentication.

password and password_file are mutually exclusive, and only one can be provided inside a basic_auth block.

Note
High scrape or write rates create repeated file reads when you use password_file. You can use the local.file component to read the password file and provide the content to the password attribute. This avoids repeated file reads because local.file monitors the file and reads when it changes.

`oauth2`

Name	Type	Description	Default	Required
`client_id`	`string`	OAuth2 client ID.		no
`client_secret_file`	`string`	File containing the OAuth2 client secret.		no
`client_secret`	`secret`	OAuth2 client secret.		no
`endpoint_params`	`map(string)`	Optional parameters to append to the token URL.		no
`no_proxy`	`string`	Comma-separated list of IP addresses, CIDR notations, and domain names to exclude from proxying.		no
`proxy_connect_header`	`map(list(secret))`	Specifies headers to send to proxies during CONNECT requests.		no
`proxy_from_environment`	`bool`	Use the proxy URL indicated by environment variables.	`false`	no
`proxy_url`	`string`	HTTP proxy to send requests through.		no
`scopes`	`list(string)`	List of scopes to authenticate with.		no
`token_url`	`string`	URL to fetch the token from.		no

client_secret and client_secret_file are mutually exclusive, and only one can be provided inside an oauth2 block.

Warning
Using client_secret_file causes the file to be read on every outgoing request. Use the local.file component with the client_secret attribute instead to avoid unnecessary reads.

The oauth2 block may also contain a separate tls_config sub-block.

no_proxy can contain IPs, CIDR notations, and domain names. IP and domain names can contain port numbers. proxy_url must be configured if no_proxy is configured.

proxy_connect_header should only be configured if proxy_url or proxy_from_environment are configured.

`tls_config`

Name	Type	Description	Required
`ca_pem`	`string`	CA PEM-encoded text to validate the server with.	no
`ca_file`	`string`	CA certificate to validate the server with.	no
`cert_pem`	`string`	Certificate PEM-encoded text for client authentication.	no
`cert_file`	`string`	Certificate file for client authentication.	no
`insecure_skip_verify`	`bool`	Disables validation of the server certificate.	no
`key_file`	`string`	Key file for client authentication.	no
`key_pem`	`secret`	Key PEM-encoded text for client authentication.	no
`min_version`	`string`	Minimum acceptable TLS version.	no
`server_name`	`string`	ServerName extension to indicate the name of the server.	no

The following pairs of arguments are mutually exclusive and can’t both be set simultaneously:

ca_pem and ca_file
cert_pem and cert_file
key_pem and key_file

When configuring client authentication, both the client certificate (using cert_pem or cert_file) and the client key (using key_pem or key_file) must be provided.

"TLS10" (TLS 1.0)
"TLS11" (TLS 1.1)
"TLS12" (TLS 1.2)
"TLS13" (TLS 1.3)

Caution
If you set insecure_skip_verify to true, you disable verification of the server’s certificate chain and hostname. Alloy accepts any certificate the server presents, including expired, self-signed, or invalid certificates.

You can use insecure_skip_verify for local development, self-signed certificates, or endpoints that use a private CA outside the system trust store. If the hostname doesn’t match the certificate, set server_name instead of insecure_skip_verify. In production, use ca_file or ca_pem to trust a private CA instead of insecure_skip_verify.

Set insecure_skip_verify to true only in isolated development or testing environments where you don’t transmit sensitive data. Remove the setting before you deploy to production.

Exported fields

The following fields are exported and can be referenced by other components:

Name	Type	Description
`receiver`	`receiver`	A value that other components can use to send profiles to.

Component health

pyroscope.write is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

Debug information

pyroscope.write doesn’t expose any component-specific debug information.

Debug metrics

The following Prometheus metrics are exposed:

Name	Type	Description
`pyroscope_write_sent_bytes_total`	`counter`	Total number of compressed bytes sent to Pyroscope endpoints.
`pyroscope_write_dropped_bytes_total`	`counter`	Total number of compressed bytes dropped by Pyroscope endpoints.
`pyroscope_write_sent_profiles_total`	`counter`	Total number of profiles sent to Pyroscope endpoints.
`pyroscope_write_dropped_profiles_total`	`counter`	Total number of profiles dropped by Pyroscope endpoints.
`pyroscope_write_retries_total`	`counter`	Total number of retries to Pyroscope endpoints.
`pyroscope_write_latency`	`histogram`	Write latency for sending profiles to Pyroscope endpoints.

All metrics include an endpoint label identifying the specific endpoint URL. The pyroscope_write_latency metric includes an additional type label with the following values:

push_total: Total latency for push operations
push_endpoint: Per-endpoint latency for push operations
push_downstream: Downstream request latency for push operations
ingest_total: Total latency for ingest operations
ingest_endpoint: Per-endpoint latency for ingest operations
ingest_downstream: Downstream request latency for ingest operations

For the push_total and ingest_total types, the endpoint label is - because these series aggregate across all endpoints rather than identifying a single one.

Troubleshoot

Connection limit errors

When using pyroscope.write to push profiles to a pyroscope.receive_http component, you may encounter errors like:

"failed to push to endpoint" err="deadline_exceeded: context deadline exceeded"

This typically indicates that the receiving component has reached its TCP connection limit.

To resolve connection limit errors, first diagnose the issue, then apply one of the solutions.

Diagnose connection limit issues

Check the connection metrics on the pyroscope.receive_http component:
- pyroscope_receive_http_tcp_connections: Current number of accepted TCP connections
- pyroscope_receive_http_tcp_connections_limit: Maximum number of TCP connections allowed
If the current connections are approaching or at the limit, you need to take action.

Solution 1: Increase the connection limit

To increase the connection limit, increase the conn_limit parameter in the pyroscope.receive_http configuration:

pyroscope.receive_http "example" {
  http {
    conn_limit = 32768  // Increase from default 16384
    // ... other settings
  }
  // ... rest of configuration
}

Solution 2: Horizontal scaling

To distribute the connection load across multiple receivers, deploy multiple instances of pyroscope.receive_http behind a load balancer.

Timeout chain issues

When chaining multiple Pyroscope components such as pyroscope.write to pyroscope.receive_http to another pyroscope.write, you may encounter timeout issues that prevent retries:

"failed to push to endpoint" err="deadline_exceeded: context deadline exceeded"

Understand the problem

This issue occurs when:

The first pyroscope.write component sends profiles to pyroscope.receive_http.
The pyroscope.receive_http component forwards profiles to another pyroscope.write component.
Both pyroscope.write components have the same default remote_timeout of 10 seconds.
The request context passes from the first pyroscope.write through pyroscope.receive_http to the second pyroscope.write, maintaining the original 10-second deadline.
If the second pyroscope.write component’s downstream request takes the full 10 seconds due to a broken TCP idle connection, there’s no time left for retries.

To resolve timeout chain issues, apply one of the following solutions.

Solution 1: Increase timeout on the first pyroscope.write

To provide buffer time for retries, increase the remote_timeout on the initial pyroscope.write component:

pyroscope.write "w1" {
  endpoint {
    url = "http://pyroscope-receiver:8080"
    remote_timeout = "30s"  // Increased from default 10s
    // ... other settings
  }
  // ... rest of configuration
}

Solution 2: Decrease timeout on the downstream pyroscope.write

To ensure faster failures and allow time for retries, reduce the remote_timeout on the downstream pyroscope.write component:

pyroscope.write "w2" {
  endpoint {
    url = "http://pyroscope-backend:4040"
    remote_timeout = "3s"  // Reduced from default 10s
    // ... other settings
  }
  // ... rest of configuration
}

Important considerations

Normal latency: Pyroscope servers with the new architecture typically have 500-1000ms average latency for requests
Timeout buffer: Always leave sufficient buffer time for retries when chaining components
Retry configuration: Consider adjusting max_backoff_retries and backoff periods alongside timeout values
Monitoring: Monitor the pyroscope_write_latency metric to understand actual request latencies and adjust timeouts accordingly

Example

pyroscope.write "staging" {
  // Send profiles to a locally running Pyroscope instance.
  endpoint {
    url = "http://pyroscope:4040"
    headers = {
      "X-Scope-OrgID" = "squad-1",
    }
  }
  external_labels = {
    "env" = "staging",
  }
}

pyroscope.scrape "default" {
  targets = [
    {"__address__" = "pyroscope:4040", "service_name"="pyroscope"},
    {"__address__" = "alloy:12345", "service_name"="alloy"},
  ]
  forward_to = [pyroscope.write.staging.receiver]
}

Compatible components

pyroscope.write has exports that can be consumed by the following components:

Components that consume Pyroscope ProfilesReceiver

Note
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.