Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

Documentation Grafana Agent Grafana Agent Flow Reference Components loki.source.file

loki.source.file

loki.source.file reads log entries from files and forwards them to other loki.* components.

Multiple loki.source.file components can be specified by giving them different labels.

Usage

river Copy

loki.source.file "LABEL" {
  targets    = TARGET_LIST
  forward_to = RECEIVER_LIST
}

Arguments

The component starts a new reader for each of the given targets and fans out log entries to the list of receivers passed in forward_to.

loki.source.file supports the following arguments:

Blocks

The loki.source.file component doesn’t support any inner blocks and is configured fully through arguments.

Exported fields

loki.source.file does not export any fields.

Component health

loki.source.file is only reported as unhealthy if given an invalid configuration.

Debug information

loki.source.file exposes some target-level debug information per reader:

• The tailed path.
• Whether the reader is currently running.
• What is the last recorded read offset in the positions file.

Debug metrics

• loki_source_file_read_bytes_total (gauge): Number of bytes read.
• loki_source_file_file_bytes_total (gauge): Number of bytes total.
• loki_source_file_read_lines_total (counter): Number of lines read.
• loki_source_file_encoding_failures_total (counter): Number of encoding failures.
• loki_source_file_files_active_total (gauge): Number of active files.

Component behavior

Each element in the list of targets as a set of key-value pairs called labels. The set of targets can either be static, or dynamically provided periodically by a service discovery component. The special label __path__ must always be present and must point to the absolute path of the file to read from.

The __path__ value is available as the filename label to each log entry the component reads. All other labels starting with a double underscore are considered internal and are removed from the log entries before they’re passed to other loki.* components.

The component uses its data path (a directory named after the domain’s fully qualified name) to store its positions file. The positions file is used to store read offsets, so that in case of a component or Agent restart, loki.source.file can pick up tailing from the same spot.

If a file is removed from the targets list, its positions file entry is also removed. When it’s added back on, loki.source.file starts reading it from the beginning.

Example

This example collects log entries from the files specified in the targets argument and forwards them to a loki.write component so they are can be written to Loki.

river Copy

loki.source.file "tmpfiles" {
  targets    = [
    {__path__ = "/tmp/foo.txt", "color" = "pink"},
    {__path__ = "/tmp/bar.txt", "color" = "blue"},
    {__path__ = "/tmp/baz.txt", "color" = "grey"},
  ] 
  forward_to = [loki.write.local.receiver]
}

loki.write "local" {
  endpoint {
    url = "loki:3100/api/v1/push"
  }
}