OpenTelemetry OpenTelemetry Collector Send logs to Loki Send logs to Loki with filelog receiver

Send logs to Loki with filelog receiver

In this guide we will show you how to collect system logs with a filelog receiver and send them to Loki via OpenTelemetry Collector using Loki exporter.

Prerequisites

Before we start, we need to have Loki and Grafana running. In this guide we will run them using docker-compose. Here is a docker-compose.yaml file:

version: "3"

networks:
  loki:

services:
  loki:
    image: grafana/loki:latest
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki

  grafana:
    image: grafana/grafana:latest
    environment:
      - "GF_AUTH_DISABLE_LOGIN_FORM=true"
      - "GF_AUTH_ANONYMOUS_ENABLED=true"
      - "GF_AUTH_ANONYMOUS_ORG_ROLE=Admin"
    ports:
      - "3000:3000"
    networks:
      - loki

Run:

docker-compose up -d

Now Grafana is running on http://localhost:3000 and Loki is running on http://localhost:3100. To visualize Loki logs in Grafana you will need to add a Loki data source. To do so open http://localhost:3000. Go to Configuration > Data sources, and click Add datasource. Choose Loki from the list of data sources. Input http://loki:3100 as a HTTP URL, and click Save & Test. The data source should be successfully added.

Download the last version of OpenTelemetry Collector

You can download the binary for most architectures/OS on GitHub for v0.63.1 or later versions, if available. The examples in this guide are based on Mac OS:

$ curl -O -L "https://github.com/open-telemetry/opentelemetry-collector-releases/releases/download/v0.76.1/otelcol-contrib_0.76.1_darwin_amd64.tar.gz"
$ gunzip -c otelcol-contrib_0.76.1_darwin_amd64.tar.gz | tar xopf -

The Filelog receiver tails and parses logs from files. In this example, we will tail system logs and send them to Grafana Loki.

Prepare OpenTelemetry Collector config file

Create a config.yaml file:

receivers:
  filelog:
    include: [/var/log/*.log]

processors:
 attributes:
   actions:
     - action: insert
       key: loki.attribute.labels
       value: log.file.name

exporters:
  loki:
    endpoint: "http://localhost:3100/loki/api/v1/push"
service:
  pipelines:
    logs:
      receivers: [filelog]
      processors: [attributes]
      exporters: [loki]

To convert OTLP attribute log.file.name into a Loki label, we need to use attributes procesor. Then run:

$ ./otelcol-contrib --config ./config.yaml

Now you can open Grafana, go to Explore, choose Loki, and see logs.

Summary

In this guide, we provided all the steps you need to configure the OpenTelemetry Collector to collect system logs with filelog receiver and send them to Loki.