Caution
Grafana Alloy is the new name for our distribution of the OTel collector. Grafana Agent has been deprecated and is in Long-Term Support (LTS) through October 31, 2025. Grafana Agent will reach an End-of-Life (EOL) on November 1, 2025. Read more about why we recommend migrating to Grafana Alloy.
Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.
otelcol.processor.discovery
otelcol.processor.discovery
accepts traces telemetry data from other otelcol
components. It can be paired with discovery.*
components, which supply a list
of labels for each discovered target.
otelcol.processor.discovery
adds resource attributes to spans which have a hostname
matching the one in the __address__
label provided by the discovery.*
component.
Note
otelcol.processor.discovery
is a custom component unrelated to any processors from the OpenTelemetry Collector.
Multiple otelcol.processor.discovery
components can be specified by giving them
different labels.
Note
It can be difficult to follow OpenTelemetry semantic conventions when adding resource attributes via
otelcol.processor.discovery
:
discovery.relabel
and mostdiscovery.*
processes such asdiscovery.kubernetes
can only emit Prometheus-compatible labels.- Prometheus labels use underscores (
_
) in labels names, whereas OpenTelemetry semantic conventions use dots (.
).- Although
otelcol.processor.discovery
is able to work with non-Prometheus labels such as ones containing dots, the fact thatdiscovery.*
components are generally only compatible with Prometheus naming conventions makes it hard to follow OpenTelemetry semantic conventions inotelcol.processor.discovery
.If your use case is to add resource attributes which contain Kubernetes metadata, consider using
otelcol.processor.k8sattributes
instead.The main use case for
otelcol.processor.discovery
is for users who migrate to Grafana Agent Flow mode from Static mode’sprom_sd_operation_type
/prom_sd_pod_associations
configuration options.
Usage
otelcol.processor.discovery "LABEL" {
targets = [...]
output {
traces = [...]
}
}
Arguments
otelcol.processor.discovery
supports the following arguments:
Name | Type | Description | Default | Required |
---|---|---|---|---|
targets | list(map(string)) | List of target labels to apply to the spans. | yes | |
operation_type | string | Configures whether to update a span’s attribute if it already exists. | upsert | no |
pod_associations | list(string) | Configures how to decide the hostname of the span. | ["ip", "net.host.ip", "k8s.pod.ip", "hostname", "connection"] | no |
targets
could come from discovery.*
components:
- The
__address__
label will be matched against the IP address of incoming spans.- If
__address__
contains a port, it is ignored.
- If
- If a match is found, then relabeling rules are applied.
- Note that labels starting with
__
will not be added to the spans.
- Note that labels starting with
The supported values for operation_type
are:
insert
: Inserts a new resource attribute if the key does not already exist.update
: Updates a resource attribute if the key already exists.upsert
: Either inserts a new resource attribute if the key does not already exist, or updates a resource attribute if the key does exist.
The supported values for pod_associations
are:
ip
: The hostname will be sourced from anip
resource attribute.net.host.ip
: The hostname will be sourced from anet.host.ip
resource attribute.k8s.pod.ip
: The hostname will be sourced from ak8s.pod.ip
resource attribute.hostname
: The hostname will be sourced from ahost.name
resource attribute.connection
: The hostname will be sourced from the context from the incoming requests (gRPC and HTTP).
If multiple pod_associations
methods are enabled, the order of evaluation is honored.
For example, when pod_associations
is ["ip", "net.host.ip"]
, "net.host.ip"
may be matched
only if "ip"
has not already matched.
Blocks
The following blocks are supported inside the definition of
otelcol.processor.discovery
:
Hierarchy | Block | Description | Required |
---|---|---|---|
output | output | Configures where to send received telemetry data. | yes |
output block
The output
block configures a set of components to forward resulting telemetry data to.
The following arguments are supported:
Name | Type | Description | Default | Required |
---|---|---|---|---|
traces | list(otelcol.Consumer) | List of consumers to send traces to. | [] | no |
You must specify the output
block, but all its arguments are optional.
By default, telemetry data is dropped.
Configure the traces
argument accordingly to send telemetry data to other components.
Exported fields
The following fields are exported and can be referenced by other components:
Name | Type | Description |
---|---|---|
input | otelcol.Consumer | A value that other components can use to send telemetry data to. |
input
accepts otelcol.Consumer
OTLP-formatted data for telemetry signals of these types:
- traces
Component health
otelcol.processor.discovery
is only reported as unhealthy if given an invalid
configuration.
Debug information
otelcol.processor.discovery
does not expose any component-specific debug
information.
Examples
Basic usage
discovery.http "dynamic_targets" {
url = "https://example.com/scrape_targets"
refresh_interval = "15s"
}
otelcol.processor.discovery "default" {
targets = discovery.http.dynamic_targets.targets
output {
traces = [otelcol.exporter.otlp.default.input]
}
}
Using more than one discovery process
Outputs from more than one discovery process can be combined via the concat
function.
discovery.http "dynamic_targets" {
url = "https://example.com/scrape_targets"
refresh_interval = "15s"
}
discovery.kubelet "k8s_pods" {
bearer_token_file = "/var/run/secrets/kubernetes.io/serviceaccount/token"
namespaces = ["default", "kube-system"]
}
otelcol.processor.discovery "default" {
targets = concat(discovery.http.dynamic_targets.targets, discovery.kubelet.k8s_pods.targets)
output {
traces = [otelcol.exporter.otlp.default.input]
}
}
Using a preconfigured list of attributes
It is not necessary to use a discovery component. In the example below, both a test_label
and
a test.label.with.dots
resource attributes will be added to a span if its IP address is
“1.2.2.2”. The __internal_label__
will be not be added to the span, because it begins with
a double underscore (__
).
otelcol.processor.discovery "default" {
targets = [{
"__address__" = "1.2.2.2",
"__internal_label__" = "test_val",
"test_label" = "test_val2",
"test.label.with.dots" = "test.val2.with.dots"}]
output {
traces = [otelcol.exporter.otlp.default.input]
}
}