Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Clients

Promtail

Stages

logfmt

Open source

logfmt

The logfmt stage is a parsing stage that reads the log line as logfmt and allows extraction of data into labels.

Schema

logfmt:
  # Set of key/value pairs for mapping of logfmt fields to extracted labels. The YAML key will be
  # the key in the extracted data, while the expression will be the YAML value. If the value
  # is empty, then the logfmt field with the same name is extracted.
  mapping:
    [ <string>: <string> ... ]

  # Name from extracted data to parse. If empty, uses the log message.
  [source: <string>]

This stage uses the go-logfmt unmarshaler, which means non-string types like numbers or booleans will be unmarshaled into those types. The extracted data can hold non-string values, and this stage does not do any type conversions; downstream stages will need to perform correct type conversion of these values as necessary. Please refer to the template stage for how to do this.

If the value extracted is a complex type, its value is extracted as a string.

Examples

Using log line

For the given pipeline:

- logfmt:
    mapping:
      timestamp: time
      app:
      duration:
      unknown:

Given the following log line:

time=2012-11-01T22:08:41+00:00 app=loki level=WARN duration=125 message="this is a log line" extra="user=foo""

The following key-value pairs would be created in the set of extracted data:

timestamp: 2012-11-01T22:08:41+00:00
app: loki
duration: 125

Using extracted data

For the given pipeline:

- logfmt:
    mapping:
      extra:
- logfmt:
    mapping:
      user:
    source: extra

And the given log line:

time=2012-11-01T22:08:41+00:00 app=loki level=WARN duration=125 message="this is a log line" extra="user=foo"

The first stage would create the following key-value pairs in the set of extracted data:

extra: user=foo

The second stage will parse the value of extra from the extracted data as logfmt and append the following key-value pairs to the set of extracted data: