This is documentation for the next version of Grafana Beyla documentation. For the latest stable release, go to the latest version.

Documentation Beyla Configure Export data

Configure Beyla Prometheus and OpenTelemetry data export

Beyla can export OpenTelemetry metrics and traces to a OTLP endpoint.

To send metrics directly to the Grafana Cloud OpenTelemetry endpoint, see the Grafana Cloud OTLP endpoint configuration.

Grafana Cloud OTLP endpoint

YAML section: grafana.otlp

You can configure the component under the grafana.otlp section of your YAML configuration or via environment variables. You can configure Beyla to submit OpenTelemetry data to the Grafana Cloud OTEL endpoint using custom variables, allowing an easier setup of the endpoint and the authentication.

For more information on the Grafana Cloud OTLP endpoint, refer to the Send data to the Grafana Cloud OTLP endpoint documentation.

For example:

YAML Copy

grafana:
  otlp:
    cloud_submit: ["metrics", "traces"]
    cloud_zone: "eu-west-23"
    cloud_instance_id: "12345"
    cloud_api_key: "abcde"

You can also use the standard OpenTelemetry variables to submit the metrics and traces to any standard OpenTelemetry endpoint, including Grafana Cloud.

Cloud zone configuration

If any of the OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_METRICS_ENDPOINT or OTEL_EXPORTER_OTLP_TRACES_ENDPOINT variables are defined, they override the destination endpoint, and Beyla ignores the cloud_zone configuration option.

OpenTelemetry metrics exporter component

YAML section: otel_metrics_export

Enable the OpenTelemetry metrics export component by setting the endpoint attribute in your configuration file or via an environment variable, refer to metric export configuration options.

Configure the component under the otel_metrics_export section of your YAML configuration or via environment variables.

In addition to the configuration documented in this article, the component supports environment variables from the standard OpenTelemetry exporter configuration.

For example:

YAML Copy

otel_metrics_export:
  ttl: 5m
  endpoint: http://otelcol:4318
  protocol: grpc
  features: ["network", "network_inter_zone"]
  buckets:
    duration_histogram: [0, 1, 2]
  histogram_aggregation: base2_exponential_bucket_histogram

Metrics export protocol

If you don’t set a protocol Beyla sets the protocol as follows:

• grpc: if the port ends in 4317, for example 4317, 14317, or 24317.
• http/protobuf: if the port ends in 4318, for example 4318, 14318, or 24318.

Metrics export features

The Beyla metrics exporter can export the following metrics data groups for processes matching entries in the metrics discovery configuration.

• application: Application RED metrics and HTTP body-size histograms

• application_red: Application RED metrics without HTTP body-size histograms

• application_sizes: HTTP request and response body-size histograms. Requires application_red in the same feature list; use application to enable both

• application_span: Application-level trace span metrics using legacy naming conventions. Deprecated: use application_span_otel instead. The name keeps working, but Beyla logs a deprecation warning at startup and the feature is removed in a future release

• application_span_otel: Application-level trace span metrics using OpenTelemetry naming conventions. Refer to span metrics formats for differences between application_span and application_span_otel

• application_span_sizes: Application-level request and response body size metrics for trace spans. Deprecated: there is no direct replacement. The closest equivalents are the http.server.request.body.size and http.server.response.body.size metrics

• application_host: Application-level host metrics for host based pricing

• application_service_graph: Application-level service graph metrics. It’s recommended to use a DNS for service discovery and to ensure the DNS names match the OpenTelemetry service names Beyla uses. In Kubernetes environments, the OpenTelemetry service name set by the service name discovery is the best choice for service graph metrics.

• application_service_graph_connection_info: An opt-in presence gauge named traces_service_graph_connection_info. It is independent of service graph request and latency metrics, is not enabled by default, and is not included in application, application_service_graph, or all.

  Beyla sets the gauge to 1 for an observed client-to-destination edge when the client service and destination identities are available. A matching server span is not required. The gauge remains active for 15 minutes after the last observation.

  Enable it in the global metrics.features list:

  YAML Copy

  metrics:
    features:
      - application_service_graph_connection_info
  prometheus_export:
    port: 8999

  This metric serves a similar topology-discovery purpose to the Tempo connection information metric, but Beyla derives it from observed client traffic. Beyla exports the metric directly; it does not require enabling the Tempo service-graphs-connection-info processor.

• application_process: Low-level process metrics (for example, CPU, memory, disk metrics) for the selected services

• application_runtime: Go, JVM, Node.js, Python, and .NET runtime metrics. Refer to Application runtime metrics for collection requirements and metric descriptions.

• network: Network-level metrics, refer to the network metrics configuration documentation to learn more

• network_flow_packets: Network packet counts. Enables beyla.network.flow.packets (beyla_network_flow_packets_total in Prometheus)

• network_inter_zone: Network inter-zone metrics, refer to the network metrics configuration documentation to learn more

• stats: All TCP statistics metrics, including the higher-frequency stats_tcp_io probes

• stats_tcp_successful_connections: Successful TCP connection counts

• stats_tcp_failed_connections: Failed TCP connection counts. Enables beyla.stat.tcp.failed.connections (beyla_stat_tcp_failed_connections_total in Prometheus)

• stats_tcp_io: Bytes sent and received at the TCP socket layer. Enables beyla.stat.tcp.io (beyla_stat_tcp_io_bytes_total in Prometheus)

• stats_tcp_retransmits: TCP retransmission counts. Enables beyla.stat.tcp.retransmits (beyla_stat_tcp_retransmits_total in Prometheus)

• stats_tcp_rtt: Smoothed TCP round-trip time. Enables the beyla.stat.tcp.rtt histogram (beyla_stat_tcp_rtt_seconds in Prometheus)

The stats feature enables every TCP statistics group. The stats_tcp_io probes run for every tcp_sendmsg and tcp_cleanup_rbuf call, so their overhead grows with TCP send and receive activity. If you don’t need TCP I/O byte counts, select the lower-frequency TCP statistics individually instead of using stats.

Successful TCP connections

Select stats_tcp_successful_connections to export beyla.stat.tcp.successful.connections (beyla_stat_tcp_successful_connections_total in Prometheus). The stats bundle includes this counter.

YAML Copy

metrics:
  features: ["application", "stats_tcp_successful_connections"]

The counter records successful TCP handshakes rather than application requests. The same feature is accepted in per-service and deprecated exporter feature lists. Use attribute selection to split connections by handshake role.

Application runtime metrics

Enable application_runtime to collect runtime metrics for instrumented Go, Java, Node.js, Python, and .NET processes. It is disabled by default. Configure it globally or per service through metrics features.

Go and Python metrics use eBPF collection. Node.js metrics require the injected Node.js agent. JVM memory metrics use HotSpot USDT probes; the Java agent supplies class, thread, CPU, and GC duration metrics. .NET metrics use the runtime diagnostic socket and EventPipe.

Go runtime metrics

JVM runtime metrics

JVM memory pool metrics use eBPF USDT probes on libjvm.so (hotspot:mem__pool__gc__begin and hotspot:mem__pool__gc__end). They remain available when javaagent.enabled is false. The other JVM metrics require the Java agent.

JVM memory metrics carry the attributes jvm.memory.type (for example, heap, non_heap) and jvm.memory.pool.name (for example, G1 Eden Space).

To control how frequently JVM GC events are sampled, refer to JVM runtime metrics.

Node.js runtime metrics

Set nodejs.enabled to true and enable application_runtime to collect these metrics. Disabling the agent disables their collection.

Python runtime metrics

.NET runtime metrics

Refer to .NET runtime collection for prerequisites and timing controls, and runtime GC buckets for JVM and V8 histograms.

Runtime exporters use fixed language-specific dimensions, including the following:

These runtime dimensions are fixed by the collectors. Per-metric attributes.select lists apply to the application, network, and TCP metrics described in attribute selection.

Span metrics formats

Beyla provides two formats for span metrics:

• application_span: Legacy format using the metric names traces_spanmetrics_latency and traces_spanmetrics_calls_total. Deprecated: use application_span_otel instead
• application_span_otel: OpenTelemetry-compliant format using the metric names traces_span_metrics_duration_seconds and traces_span_metrics_calls_total

You can only enable one span metrics format at a time. If you specify both application_span and application_span_otel in the features list, Beyla returns a configuration error. The same applies when one format comes from the top-level metrics section and the other from a per-service discovery metrics.features section, because the exported metric names are selected from the combination of both.

If you select the formats implicitly with all or *, Beyla resolves the conflict automatically in favor of application_span_otel and logs a warning. This resolution also applies to per-service metrics.features sections, so a single service requesting all doesn’t move every service back to the legacy names.

Metrics instrumentation

The list of instrumentation areas Beyla can collection data from:

• *: all instrumentation, if * is present Beyla ignores other values
• aerospike: Aerospike database metrics
• amqp: AMQP message queue metrics
• couchbase: Couchbase database metrics
• dns: DNS request metrics
• genai: Generative AI metrics
• gpu: GPU operation metrics
• http: HTTP/HTTPS/HTTP2 application metrics
• grpc: gRPC application metrics
• kafka: Kafka client/server message queue metrics
• memcached: Memcached database metrics
• mongo: MongoDB client/server database metrics
• mqtt: MQTT message queue metrics
• nats: NATS message queue metrics
• redis: Redis client/server database metrics
• sql: SQL database client call metrics
• sunrpc: SunRPC call metrics

For example, setting the instrumentations option to: http,grpc enables the collection of HTTP/HTTPS/HTTP2 and gRPC application metrics, and disables other instrumentation.

OpenTelemetry traces exporter component

YAML section: otel_traces_export

You can configure the component under the otel_traces_export section of your YAML configuration or via environment variables.

In addition to the configuration documented in this article, the component supports the environment variables from the standard OpenTelemetry exporter configuration.

YAML Copy

otel_traces_export:
  endpoint: http://jaeger:4317
  protocol: grpc
  instrumentations: ["http", "sql"]

Traces export protocol

If you don’t set a protocol Beyla sets the protocol as follows:

• grpc: if the port ends in 4317, for example 4317, 14317, or 24317.
• http/protobuf: if the port ends in 4318, for example 4318, 14318, or 24318.

Traces export compression

Set otel_traces_export.compression to gzip to compress OTLP trace payloads, or none to send them uncompressed. The default is uncompressed. Gzip reduces transfer size at the cost of compression CPU time and is supported by the HTTP and gRPC exporters.

YAML Copy

otel_traces_export:
  endpoint: http://collector:4317
  protocol: grpc
  compression: gzip

OTEL_EXPORTER_OTLP_TRACES_COMPRESSION sets the traces-specific value and overrides YAML. When no traces-specific value is set, OTEL_EXPORTER_OTLP_COMPRESSION supplies the shared value. Both accept none and gzip.

For example, compress traces while keeping the shared setting uncompressed:

sh Copy

export OTEL_EXPORTER_OTLP_COMPRESSION=none
export OTEL_EXPORTER_OTLP_TRACES_COMPRESSION=gzip

Traces instrumentation

The list of instrumentation areas Beyla can collection data from:

• *: all instrumentation, if * is present Beyla ignores other values
• aerospike: Aerospike database traces
• amqp: AMQP message queue traces
• couchbase: Couchbase database traces
• dns: DNS request traces
• genai: Generative AI traces
• gpu: GPU operation traces
• http: HTTP/HTTPS/HTTP2 application traces
• grpc: gRPC application traces
• kafka: Kafka client/server message queue traces
• memcached: Memcached database traces
• mongo: MongoDB client/server database traces
• mqtt: MQTT message queue traces
• nats: NATS message queue traces
• redis: Redis client/server database traces
• sql: SQL database client call traces
• sunrpc: SunRPC call traces

For example, setting the instrumentations option to: http,grpc enables the collection of HTTP/HTTPS/HTTP2 and gRPC application traces, and disables other instrumentation.

Note: By default, Beyla enables all supported trace instrumentations except dns, genai, and gpu. You can explicitly enable these instrumentations if needed.

Prometheus exporter component

YAML section: prometheus_export

You can configure the component under the prometheus_export section of your YAML configuration or via environment variables. This component opens an HTTP endpoint in the auto-instrumentation tool that allows any external scraper to pull metrics in Prometheus format. It is enabled if the port property is set.

YAML Copy

prometheus_export:
  port: 8999
  path: /metrics
  extra_resource_attributes: ["deployment_environment"]
  ttl: 1s
  buckets:
    request_size_histogram: [0, 10, 20, 22]
    response_size_histogram: [0, 10, 20, 22]
  features:
    - application
    - network
    - application_process
    - application_span
    - application_service_graph
  instrumentations: ["http, "sql"]

Prometheus extra resource attributes

Due to internal limitations of the Prometheus API client, Beyla needs to know beforehand which attributes are exposed for each metric. This would cause that some attributes that are discovered at runtime, during instrumentation, won’t be visible by default. For example, attributes defined on each application via Kubernetes annotations, or in the target application’s OTEL_RESOURCE_ATTRIBUTES environment variable.

For example, an application defining the OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production as environment variable, the target_info{deployment.environment="production"} attribute would be visible by default if the metrics are exported via OpenTelemetry but not if they are exported via Prometheus.

To make deployment_environment visible in Prometheus, you need to add it to the extra_resource_attributes list.

Prometheus export features

The Prometheus metrics exporter can export the following metrics data groups:

• application: Application RED metrics and HTTP body-size histograms
• application_red: Application RED metrics without HTTP body-size histograms
• application_sizes: HTTP request and response body-size histograms
• application_span: Application-level trace span metrics
• application_host Application-level host metrics for host based pricing
• application_service_graph: Application-level service graph metrics. It’s recommended to use a DNS for service discovery and to ensure the DNS names match the OpenTelemetry service names Beyla uses. In Kubernetes environments, the OpenTelemetry service name set by the service name discovery is the best choice for service graph metrics.
• application_process: Low-level process metrics (for example, CPU, memory, disk metrics) for the selected services
• application_runtime: Go, JVM, Node.js, Python, and .NET runtime metrics. Refer to Application runtime metrics for collection requirements and metric descriptions.
• network: Network-level metrics, refer to the network metrics configuration documentation to learn more
• network_flow_packets: Network packet counts
• network_inter_zone: Network inter-zone metrics, refer to the network metrics configuration documentation to learn more
• stats: All TCP statistics metrics
• stats_tcp_successful_connections: Successful TCP connection counts
• stats_tcp_failed_connections: Failed TCP connection counts
• stats_tcp_io: Bytes sent and received at the TCP socket layer
• stats_tcp_retransmits: TCP retransmission counts
• stats_tcp_rtt: Smoothed TCP round-trip time

The prometheus_export.features option is deprecated. Use the global metrics.features option for new configurations.

Prometheus instrumentation

The list of instrumentation areas Beyla can collection data from:

• *: all instrumentation, if * is present Beyla ignores other values
• aerospike: Aerospike database metrics
• amqp: AMQP message queue metrics
• couchbase: Couchbase database metrics
• dns: DNS request metrics
• genai: Generative AI metrics
• gpu: GPU operation metrics
• http: HTTP/HTTPS/HTTP2 application metrics
• grpc: gRPC application metrics
• kafka: Kafka client/server message queue metrics
• memcached: Memcached database metrics
• mongo: MongoDB client/server database metrics
• mqtt: MQTT message queue metrics
• nats: NATS message queue metrics
• redis: Redis client/server database metrics
• sql: SQL database client call metrics
• sunrpc: SunRPC call metrics

For example, setting the instrumentations option to: http,grpc enables the collection of HTTP/HTTPS/HTTP2 and gRPC application metrics, and disables other instrumentation.

Global metrics features

You can set the default features list for all exporters at once using the global metrics configuration block, rather than repeating it under each exporter. Deprecated exporter feature settings can override this global setting for both exporters.

The metrics.features option (environment variable: BEYLA_METRICS_FEATURES, also accepted as OTEL_EBPF_METRICS_FEATURES) accepts a list of feature group names. The default is ["application"].

You can also override the global features for one discovered service with discovery.instrument[].metrics.features. The deprecated exporter-specific options also override the global list. When the OTLP metrics endpoint is enabled and otel_metrics_export.features is configured, that list wins. Otherwise, an enabled Prometheus endpoint with prometheus_export.features supplies the list. This normalized list applies to both exporters. Use metrics.features for new configurations.

application enables both application_red and application_sizes. The size group exports http.server.request.body.size, http.server.response.body.size, http.client.request.body.size, and http.client.response.body.size. To collect RED metrics without these four histograms:

YAML Copy

metrics:
  features: ["application_red"]

To collect both groups explicitly:

YAML Copy

metrics:
  features: ["application_red", "application_sizes"]

The same values are accepted in discovery.instrument[].metrics.features and the deprecated exporter lists. Per-service lists replace the global selection for the matching process. Use BEYLA_PROMETHEUS_FEATURES for the legacy Prometheus override. BEYLA_OTEL_METRICS_FEATURES is a compatibility alias for global metrics.features; the legacy OTLP exporter override is configured through YAML.

To collect application and runtime metrics for all instrumented services:

YAML Copy

metrics:
  features: ["application", "application_runtime"]

To enable selected network and TCP statistics globally for all processes:

YAML Copy

metrics:
  features:
    - application
    - network_flow_packets
    - stats_tcp_failed_connections
    - stats_tcp_retransmits
    - stats_tcp_rtt

To enable application_runtime only for a specific process, use the per-service metrics.features override in service discovery:

YAML Copy

discovery:
  instrument:
    - name: my-java-service
      exe_path: "*/java"
      metrics:
        features:
          - application_runtime

JVM runtime metrics

When application_runtime is enabled for Java services, Beyla collects memory pool metrics from HotSpot USDT probes and other JVM metrics through the Java agent. Set javaagent.enabled to false to disable agent collection while keeping HotSpot memory metrics.

The sampling interval controls HotSpot memory event sampling and Java agent class, thread, and CPU snapshots. It limits collection frequency for JVM processes with frequent GC events.

Example:

YAML Copy

jvm_runtime_metrics:
  sampling_interval: 1s

Collect .NET runtime metrics

This collection requires .NET 8 or later. Enable application_runtime for the target service. Beyla connects to the .NET diagnostic socket in the process’s temporary directory, using TMPDIR when configured or /tmp otherwise. Diagnostics must be enabled and the socket must be accessible to Beyla.

For example:

YAML Copy

metrics:
  features: ["application", "application_runtime"]
dotnet_runtime_metrics:
  sampling_interval: 2s
  timeout: 15s

Processes with the same service identity contribute to shared runtime series. Beyla tracks counter baselines per process; exported counters describe the observations available to the collector.