Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Documentation Grafana Alloy Reference Components otelcol otelcol.receiver.filelog

otelcol.receiver.filelog

otelcol.receiver.filelog reads log entries from files and forwards them to other otelcol.* components.

You can specify multiple otelcol.receiver.filelog components by giving them different labels.

Usage

Alloy Copy

otelcol.receiver.filelog "<LABEL>" {
  include = [...]
  output {
    logs    = [...]
  }
}

Arguments

You can use the following arguments with otelcol.receiver.filelog:

encoding must be one of utf-8, utf-16le, utf-16be, ascii, big5, or nop. Refer to the upstream receiver documentation for more details.

start_at must be one of beginning or end. The header block may only be used if start_at is beginning.

compression must be either `` or gzip.

To persist state between restarts of the Alloy process, set the storage attribute to the handler exported from an otelcol.storage.* component.

operators

The operators list is a list of stanza operators that transform the log entries after they have been read.

For example, if container logs are being collected you may want to utilize the stanza container parser operator to add relevant attributes to the log entries.

Alloy Copy

otelcol.receiver.filelog "default" {
    ...
    operators = [
      {
        type = "container"
      }
    ]
}

Blocks

You can use the following blocks with otelcol.receiver.filelog:

The > symbol indicates deeper levels of nesting. For example, ordering_criteria > sort_by refers to a sort_by block defined inside a ordering_criteria block.

output

debug_metrics

header

The header block configures logic for parsing a log header line into additional attributes added to each log entry. It may only be used when start_at is set to beginning. The following arguments are supported:

If a header block isn’t set, no log lines will be treated as header metadata.

The metadata_operators list is a list of stanza operators that parses metadata from the header. Any attributes created from the embedded operators pipeline will be applied to all log entries in the file.

For example, you might use a regex_parser to process a header line that has been identified by the pattern expression. The following example shows a fictitious header line, and then the header block that would parse an environment attribute from it.

text Copy

HEADER_IDENTIFIER env="production"
...

Alloy Copy

otelcol.receiver.filelog "default" {
    ...
    header {
      pattern = '^HEADER_IDENTIFIER .*$'
      metadata_operators = [
        {
          type = "regex_parser"
          regex = 'env="(?P<environment>.+)"'
        }
      ]
    }
}

multiline

The multiline block configures logic for splitting incoming log entries. The following arguments are supported:

A multiline block must contain either line_start_pattern or line_end_pattern.

If a multiline block isn’t set, log entries won’t be split.

ordering_criteria

The ordering_criteria block configures the order in which log files discovered will be processed. The following arguments are supported:

sort_by

The sort_by repeatable block configures the way the fields parsed in the ordering_criteria block will be applied to sort the discovered log files. The following arguments are supported:

sort_type must be one of numeric, lexicographic, timestamp, or mtime. When using numeric, lexicographic, or timestamp sort_type, a named capture group defined in the regex attribute in ordering_criteria must be provided in regex_key. When using mtime sort_type, the file’s modified time will be used to sort.

The location and layout arguments are only applicable when sort_type is timestamp.

The location argument specifies a Time Zone identifier. The available locations depend on the local IANA Time Zone database. Refer to the list of tz database time zones in Wikipedia for a non-comprehensive list.

retry_on_failure

The retry_on_failure block configures the retry behavior when the receiver encounters an error downstream in the pipeline. A backoff algorithm is used to delay the retry upon subsequent failures. The following arguments are supported:

If max_elapsed_time is set to 0 data is never discarded.

Exported fields

otelcol.receiver.filelog doesn’t export any fields.

Component health

otelcol.receiver.filelog is only reported as unhealthy if given an invalid configuration.

Debug metrics

otelcol.receiver.filelog doesn’t expose any component-specific debug metrics.

Example

This example reads log entries using the otelcol.receiver.filelog receiver and they’re logged by a otelcol.exporter.debug component. It expects the logs to start with an ISO8601 compatible timestamp and parses it from the log using the regex_parser operator.

Alloy Copy

otelcol.receiver.filelog "default" {
  include = ["/var/log/*.log"]
  operators = [{
    type = "regex_parser",
    regex = "^(?P<timestamp>\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{3,6}Z)",
    timestamp = {
      parse_from = "attributes.timestamp",
      layout = "%Y-%m-%dT%H:%M:%S.%fZ",
      location = "UTC",
    },
  }]
  output {
      logs = [otelcol.exporter.debug.default.input]
  }
}

otelcol.exporter.debug "default" {}

Compatible components

otelcol.receiver.filelog can accept arguments from the following components:

• Components that export OpenTelemetry otelcol.Consumer