Menu

This is documentation for the next version of Agent. For the latest stable release, go to the latest version.

Experimental

otelcol.processor.filter

EXPERIMENTAL: This is an experimental component. Experimental components are subject to frequent breaking changes, and may be removed with no equivalent replacement.

otelcol.processor.filter accepts and filters telemetry data from other otelcol components using the OpenTelemetry Transformation Language (OTTL). If any of the OTTL statements evaluates to true, the telemetry data is dropped.

OTTL statements consist of OTTL Converter functions, which act on paths. A path is a reference to a telemetry data such as:

  • Resource attributes.
  • Instrumentation scope name.
  • Span attributes.

In addition to the standard OTTL Converter functions, the following metrics-only functions are used exclusively by the processor:

OTTL statements used in otelcol.processor.filter mostly contain constructs such as:

  • Booleans:
    • not true
    • not IsMatch(name, "http_.*")
  • Math expressions:
    • 1 + 1
    • end_time_unix_nano - start_time_unix_nano
    • sum([1, 2, 3, 4]) + (10 / 1) - 1

Note

Raw River strings can be used to write OTTL statements. For example, the OTTL statement attributes["grpc"] == true is written in River as `attributes[“grpc”] == true`

Note

otelcol.processor.filter is a wrapper over the upstream OpenTelemetry Collector filter processor. If necessary, bug reports or feature requests will be redirected to the upstream repository.

You can specify multiple otelcol.processor.filter components by giving them different labels.

Warning

Exercise caution when using otelcol.processor.filter:

  • Make sure you understand schema/format of the incoming data and test the configuration thoroughly. In general, use a configuration that is as specific as possible ensure you retain only the data you want to keep.
  • Orphaned Telemetry: The processor allows dropping spans. Dropping a span may lead to orphaned spans if the dropped span is a parent. Dropping a span may lead to orphaned logs if the log references the dropped span.

Usage

river
otelcol.processor.filter "LABEL" {
  output {
    metrics = [...]
    logs    = [...]
    traces  = [...]
  }
}

Arguments

otelcol.processor.filter supports the following arguments:

NameTypeDescriptionDefaultRequired
error_modestringHow to react to errors if they occur while processing a statement."propagate"no

The supported values for error_mode are:

  • ignore: Ignore errors returned by statements and continue on to the next statement. This is the recommended mode.
  • propagate: Return the error up the pipeline. This will result in the payload being dropped from the Agent.

Blocks

The following blocks are supported inside the definition of otelcol.processor.filter:

HierarchyBlockDescriptionRequired
tracestracesStatements which filter traces.no
metricsmetricsStatements which filter metrics.no
logslogsStatements which filter logs.no
outputoutputConfigures where to send received telemetry data.yes

traces block

The traces block specifies statements that filter trace telemetry signals. Only one traces block can be specified.

NameTypeDescriptionDefaultRequired
spanlist(string)List of OTTL statements filtering OTLP spans.no
spaneventlist(string)List of OTTL statements filtering OTLP span events.no

The syntax of OTTL statements depends on the OTTL context. See the OpenTelemetry documentation for more information:

Statements are checked in order from “high level” to “low level” telemetry, in this order:

  1. span
  2. spanevent

If at least one span condition is satisfied, the spanevent conditions will not be checked. Only one of the statements inside the list of statements has to be satisfied.

If all span events for a span are dropped, the span will be left intact.

metrics block

The metrics block specifies statements that filter metric telemetry signals. Only one metrics blocks can be specified.

NameTypeDescriptionDefaultRequired
metriclist(string)List of OTTL statements filtering OTLP metric.no
datapointlist(string)List of OTTL statements filtering OTLP metric datapoints.no

The syntax of OTTL statements depends on the OTTL context. See the OpenTelemetry documentation for more information:

Statements are checked in order from “high level” to “low level” telemetry, in this order:

  1. metric
  2. datapoint

If at least one metric condition is satisfied, the datapoint conditions will not be checked. Only one of the statements inside the list of statements has to be satisfied.

If all datapoints for a metric are dropped, the metric will also be dropped.

logs block

The logs block specifies statements that filter log telemetry signals. Only logs blocks can be specified.

NameTypeDescriptionDefaultRequired
log_recordlist(string)List of OTTL statements filtering OTLP metric.no

The syntax of OTTL statements depends on the OTTL context. See the OpenTelemetry documentation for more information:

Only one of the statements inside the list of statements has to be satisfied.

output block

The output block configures a set of components to forward resulting telemetry data to.

The following arguments are supported:

NameTypeDescriptionDefaultRequired
logslist(otelcol.Consumer)List of consumers to send logs to.[]no
metricslist(otelcol.Consumer)List of consumers to send metrics to.[]no
traceslist(otelcol.Consumer)List of consumers to send traces to.[]no

You must specify the output block, but all its arguments are optional. By default, telemetry data is dropped. Configure the metrics, logs, and traces arguments accordingly to send telemetry data to other components.

Exported fields

The following fields are exported and can be referenced by other components:

NameTypeDescription
inputotelcol.ConsumerA value that other components can use to send telemetry data to.

input accepts otelcol.Consumer data for any telemetry signal (metrics, logs, or traces).

Component health

otelcol.processor.filter is only reported as unhealthy if given an invalid configuration.

Debug information

otelcol.processor.filter does not expose any component-specific debug information.

Debug metrics

otelcol.processor.filter does not expose any component-specific debug metrics.

Examples

Drop spans which contain a certain span attribute

This example sets the attribute test to pass if the attribute test does not exist.

river
otelcol.processor.filter "default" {
  error_mode = "ignore"

  traces {
    span = [
      "attributes[\"container.name\"] == \"app_container_1\"",
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Each " is escaped with \" inside the River string.

Drop metrics based on either of two criteria

This example drops metrics which satisfy at least one of two OTTL statements:

  • The metric name is my.metric and there is a my_label resource attribute with a value of abc123 .
  • The metric is a histogram.
river
otelcol.processor.filter "default" {
  error_mode = "ignore"

  metrics {
    metric = [
       "name == \"my.metric\" and resource.attributes[\"my_label\"] == \"abc123\""
       "type == METRIC_DATA_TYPE_HISTOGRAM"
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Some values in the River string are escaped:

  • \ is escaped with \\
  • " is escaped with \"

Drop non-HTTP spans and sensitive logs

river
otelcol.processor.filter "default" {
  error_mode = "ignore"

  traces {
    span = [
      "attributes[\"http.request.method\"] == nil",
    ]
  }

  logs {
    log_record = [
      "IsMatch(body, \".*password.*\")",
      "severity_number < SEVERITY_NUMBER_WARN",
    ]
  }

  output {
    metrics = [otelcol.exporter.otlp.default.input]
    logs    = [otelcol.exporter.otlp.default.input]
    traces  = [otelcol.exporter.otlp.default.input]
  }
}

Each " is escaped with \" inside the River string.

Some values in the River strings are escaped:

  • \ is escaped with \\
  • " is escaped with \"

Compatible components

otelcol.processor.filter can accept arguments from the following components:

otelcol.processor.filter has exports that can be consumed by the following components:

Note

Connecting some components may not be sensible or components may require further configuration to make the connection work correctly. Refer to the linked documentation for more details.