Installation on Grafana Labs

Install Grafana Loki with Helm

Thu, 10 Apr 2025 12:15:54 +0000

Install Grafana Loki with Helm

The Helm chart allows you to configure, install, and upgrade Grafana Loki within a Kubernetes cluster.

This guide references the Loki Helm chart version 3.0 or greater and contains the following sections:

Reference

Values reference

Tanka

Wed, 06 Sep 2023 12:47:18 +0000

Tanka

Tanka is a reimplementation of Ksonnet that Grafana Labs created after Ksonnet was deprecated. Tanka is used by Grafana Labs to run Grafana Loki in production.

The Tanka installation runs the Loki cluster in microservices mode.

Prerequisites

Install the latest version of Tanka (version v0.17.1 or a more recent version) for the tk env commands. Prebuilt binaries for Tanka can be found at the Tanka releases URL.

In your config repo, if you don’t have a Tanka application, create a folder and call tk init inside of it. Then create an environment for Loki and provide the URL for the Kubernetes API server to deploy to (e.g., https://localhost:6443):

mkdir <application name>
cd <application name>
tk init
tk env add environments/loki --namespace=loki --server=<Kubernetes API server>

Install jsonnet-bundler (jb), find instructions for your platform in Tanka’s installation docs.

Deploying

Download and install the Loki and Promtail module using jb (version v0.4.0 or a more recent version):

jb init  # not required if you already ran `tk init`
jb install github.com/grafana/loki/production/ksonnet/loki@main
jb install github.com/grafana/loki/production/ksonnet/promtail@main

Revise the YAML contents of environments/loki/main.jsonnet, updating these variables:

Update the username, password, and the relevant htpasswd variable values.
Update the S3 or GCS variable values, depending on your object storage type. See storage_config for more configuration details.
Remove from the configuration the S3 or GCS object storage variables that are not part of your setup.
Update the value of boltdb_shipper_shared_store or tsdb_shipper_shared_store to the type of object storage you are using. Options are gcs, s3, azure, cos or filesystem. Update the object_store variable under the schema_config section to the same value.
Update the Promtail configuration container_root_path variable’s value to reflect your root path for the Docker daemon. Run docker info | grep "Root Dir" to acquire your root path.
Update the from value in the Loki schema_config section to no more than 14 days prior to the current date. The from date represents the first day for which the schema_config section is valid. For example, if today is 2021-01-15, set from to 2021-01-01. This recommendation is based on Loki’s default acceptance of log lines up to 14 days in the past. The reject_old_samples_max_age configuration variable controls the acceptance range.

local gateway = import 'loki/gateway.libsonnet';
local loki = import 'loki/loki.libsonnet';
local promtail = import 'promtail/promtail.libsonnet';

loki + promtail + gateway {
  _config+:: {
    namespace: 'loki',
    htpasswd_contents: 'loki:$apr1$H4yGiGNg$ssl5/NymaGFRUvxIV1Nyr.',

    // S3 variables -- Remove if not using s3
    storage_backend: 's3,dynamodb',
    s3_access_key: 'key',
    s3_secret_access_key: 'secret access key',
    s3_address: 'url',
    s3_bucket_name: 'loki-test',
    dynamodb_region: 'region',

    // GCS variables -- Remove if not using gcs
    storage_backend: 'bigtable,gcs',
    bigtable_instance: 'instance',
    bigtable_project: 'project',
    gcs_bucket_name: 'bucket',

    //Set this variable based on the type of object storage you're using.
    boltdb_shipper_shared_store: 'my-object-storage-backend-type',

    //Update the object_store and from fields
    loki+: {
      schema_config: {
        configs: [{
          from: 'YYYY-MM-DD',
          store: 'boltdb-shipper',
          object_store: 'my-object-storage-backend-type',
          schema: 'v11',
          index: {
            prefix: '%s_index_' % $._config.table_prefix,
            period: '%dh' % $._config.index_period_hours,
          },
        }],
      },
    },

    //Update the container_root_path if necessary
    promtail_config+: {
      clients: [{
        scheme:: 'http',
        hostname:: 'gateway.%(namespace)s.svc' % $._config,
        username:: 'loki',
        password:: 'password',
        container_root_path:: '/var/lib/docker',
      }],
    },

    replication_factor: 3,
    consul_replicas: 1,
  },
}

Run tk show environments/loki to see the manifests that will be deployed to the cluster. Run tk apply environments/loki to deploy the manifests. To delete the environment from cluster, run tk delete environments/loki.

Install Loki with Docker or Docker Compose

Wed, 07 Feb 2024 18:57:52 +0000

Install Loki with Docker or Docker Compose

You can install Loki and Promtail with Docker or Docker Compose if you are evaluating, testing, or developing Loki. For production, Grafana recommends installing with Tanka or Helm.

The configuration acquired with these installation instructions run Loki as a single binary.

Prerequisites

Docker
Docker Compose (optional, only needed for the Docker Compose install method)

Install with Docker

Linux

Copy and paste the commands below into your command line.

wget https://raw.githubusercontent.com/grafana/loki/v2.9.4/cmd/loki/loki-local-config.yaml -O loki-config.yaml
docker run --name loki -d -v $(pwd):/mnt/config -p 3100:3100 grafana/loki:2.9.4 -config.file=/mnt/config/loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/v2.9.4/clients/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
docker run --name promtail -d -v $(pwd):/mnt/config -v /var/log:/var/log --link loki grafana/promtail:2.9.4 -config.file=/mnt/config/promtail-config.yaml

When finished, loki-config.yaml and promtail-config.yaml are downloaded in the directory you chose. Docker containers are running Loki and Promtail using those config files.

Navigate to http://localhost:3100/metrics to view the metrics and http://localhost:3100/ready for readiness.

The image is configured to run by default as user loki with UID 10001 and GID 10001. You can use a different user, specially if you are using bind mounts, by specifying the UID with a docker run command and using --user=UID with numeric UID suited to your needs.

Windows

Copy and paste the commands below into your terminal. Note that you will need to replace the <placeholders> in the commands with your local path.

cd "<local-path>"
wget https://raw.githubusercontent.com/grafana/loki/v2.9.4/cmd/loki/loki-local-config.yaml -O loki-config.yaml
docker run --name loki -v <local-path>:/mnt/config -p 3100:3100 grafana/loki:2.9.4 --config.file=/mnt/config/loki-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/v2.9.4/clients/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml
docker run -v <local-path>:/mnt/config -v /var/log:/var/log --link loki grafana/promtail:2.9.4 --config.file=/mnt/config/promtail-config.yaml

When finished, loki-config.yaml and promtail-config.yaml are downloaded in the directory you chose. Docker containers are running Loki and Promtail using those config files.

Navigate to http://localhost:3100/metrics to view the output.

Install with Docker Compose

Run the following commands in your command line. They work for Windows or Linux systems.

wget https://raw.githubusercontent.com/grafana/loki/v2.9.4/production/docker-compose.yaml -O docker-compose.yaml
docker-compose -f docker-compose.yaml up

Local

Thu, 10 Apr 2025 12:15:54 +0000

Local

In order to log events with Grafana Loki, download and install both Promtail and Loki.

Loki is the logging engine.
Promtail sends logs to Loki.

The configuration specifies running Loki as a single binary.

Install using APT or RPM package manager

Add Granafa’s Advanced Package Tool APT or RPM Package Manager RPM package repository following the linked instructions.

Install Loki and Promtail

Using dnf
```
dnf update
dnf install loki promtail
```

Using apt-get

apt-get update
apt-get install loki promtail

Install manually

Navigate to the release page.
Scroll down to the Assets section under the version that you want to install.
Download the Loki and Promtail .zip files that correspond to your system. Note: Do not download LogCLI or Loki Canary at this time. LogCLI allows you to run Loki queries in a command line interface. Loki Canary is a tool to audit Loki performance.
Unzip the package contents into the same directory. This is where the two programs will run.
In the command line, change directory (cd on most systems) to the directory with Loki and Promtail. Copy and paste the commands below into your command line to download generic configuration files. Note: Use the corresponding Git refs that match your downloaded Loki version to get the correct configuration file. For example, if you are using Loki version 2.6.1, you need to use the https://raw.githubusercontent.com/grafana/loki/v2.9.4/cmd/loki/loki-local-config.yaml URL to download the configuration file that corresponds to the Loki version you aim to run.
```
wget https://raw.githubusercontent.com/grafana/loki/main/cmd/loki/loki-local-config.yaml
wget https://raw.githubusercontent.com/grafana/loki/main/clients/cmd/promtail/promtail-local-config.yaml
```

Enter the following command to start Loki:

Windows

.\loki-windows-amd64.exe --config.file=loki-local-config.yaml

Linux

./loki-linux-amd64 -config.file=loki-local-config.yaml

Loki runs and displays Loki logs in your command line and on http://localhost:3100/metrics.

The next step will be running an agent to send logs to Loki. To do so with Promtail, refer to the Promtail configuration.

Release binaries - openSUSE Linux only

Every release includes binaries for Loki which can be found on the Releases page.

Community openSUSE Linux packages

The community provides packages of Loki for openSUSE Linux. To install:

Add the repository https://download.opensuse.org/repositories/security:/logging/ to your system. For example, if you are using Leap 15.1, run sudo zypper ar https://download.opensuse.org/repositories/security:/logging/openSUSE_Leap_15.1/security:logging.repo ; sudo zypper ref
Install the Loki package with zypper in loki
Enable the Loki and Promtail services:
- systemd start loki && systemd enable loki
- systemd start promtail && systemd enable promtail
Modify the configuration files as needed: /etc/loki/promtail.yaml and /etc/loki/loki.yaml.

Install on Istio

Wed, 06 Sep 2023 12:47:18 +0000

Install on Istio

When installing Loki on Istio service mesh you must complete some additional steps. Without these steps, the ingester, querier, etc. might start, but you will see logs like the following:

loki level=debug ts=2021-11-24T11:33:37.352544925Z caller=broadcast.go:48 msg="Invalidating forwarded broadcast" key=collectors/distributor version=123 oldVersion=122 content=[loki-distributor-59c4896444-t9t6g[] oldContent=[loki-distributor-59c4896444-t9t6g[]

This means that the pod is failing to join the ring.

If you try to add loki to Grafana data sources, you will see logs like (empty ring)

loki level=warn ts=2021-11-24T08:02:42.08262122Z caller=logging.go:72 traceID=3fc821042d8ada1a orgID=fake msg="GET /loki/api/v1/labels?end=1637740962079859431&start=1637740361925000000 (500) 97.4µs Response: \"empty ring\\n\" ws: false; X-Scope-Orgid: fake; uber-trace-id: 3fc821042d8ada1a:1feed8872deea75c:1180f95a8235bb6c:0; "

When you enable istio-injection on the namespace where Loki is running, you need to also modify the configuration for the Loki services. Given that Istio will not allow a pod to resolve another mod using an IP address, you must also modify the memberlist service.

Required changes

Query frontend service

Make the following modifications to the file for Loki’s Query Frontend service.

Change the name of grpc port to grpclb. This is used by the grpc load balancing strategy which relies on SRV records. Otherwise the querier will not be able to reach the query-frontend. See https://github.com/grafana/loki/blob/0116aa61c86fa983ddcbbd5e30a2141d2e89081a/production/ksonnet/loki/common.libsonnet#L19 and https://grpc.github.io/grpc/core/md_doc_load-balancing.html
Set the appProtocol of grpclb to tcp
Set publishNotReadyAddresses to true

apiVersion: v1
kind: Service
metadata:
  labels:
    app: loki-query-frontend
    app.kubernetes.io/instance: observability
    app.kubernetes.io/name: loki-query-frontend
spec:
  ports:
  - appProtocol: http
    name: http
    port: 3100
    protocol: TCP
    targetPort: http
  - appProtocol: tcp
    name: grpclb
    port: 9095
    protocol: TCP
    targetPort: grpc
  publishNotReadyAddresses: true
  selector:
    app: loki-query-frontend
    app.kubernetes.io/instance: observability
    app.kubernetes.io/name: loki-query-frontend
  type: ClusterIP

Querier service

Make the following modifications to the file for Loki’s Querier service.

Set the appProtocol of the grpc service to tcp

apiVersion: v1
kind: Service
metadata:
  labels:
    app: loki-querier
    app.kubernetes.io/instance: observability
    app.kubernetes.io/name: loki-querier
  name: loki-querier
  namespace: observability
spec:
  ports:
  - appProtocol: http
    name: http
    port: 3100
    protocol: TCP
    targetPort: http
  - appProtocol: tcp
    name: grpc
    port: 9095
    protocol: TCP
    targetPort: grpc
  selector:
    app: loki-querier
    app.kubernetes.io/instance: observability
    app.kubernetes.io/name: loki-querier
  type: ClusterIP

Ingester service and Ingester headless service

Make the following modifications to the file for Loki’s Query Ingester and Ingester Headless service.

Set the appProtocol of the grpc port to tcp

apiVersion: v1
kind: Service
metadata:
  labels:
    app: loki-ingester-(headless)
    app.kubernetes.io/instance: observability
    app.kubernetes.io/name: loki-ingester
  name: loki-ingester-headless
spec:  
  clusterIP: None (if headless)
  ports:
  - name: http
    port: 3100
    protocol: TCP
    targetPort: http
  - appProtocol: tcp
    name: grpc
    port: 9095
    protocol: TCP
    targetPort: grpc
  selector:
    app: loki-ingester
    app.kubernetes.io/instance: observability
    app.kubernetes.io/name: loki-ingester
  type: ClusterIP

Distributor service

Make the following modifications to the file for Loki’s Distributor service.

Set the appProtocol of the grpc port to tcp

apiVersion: v1
kind: Service
metadata:
  labels:
    app: loki-distributor
    app.kubernetes.io/instance: observability
    app.kubernetes.io/name: loki-distributor
spec:
  ports:
  - name: http
    port: 3100
    protocol: TCP
    targetPort: http
  - name: grpc
    port: 9095
    protocol: TCP
    targetPort: grpc
    appProtocol: tcp
  selector:
    app: loki-distributor
    app.kubernetes.io/instance: observability
    app.kubernetes.io/name: loki-distributor
  sessionAffinity: None
  type: ClusterIP

Memberlist service

Make the following modifications to the file for the Memberlist service.

Set the appProtocol of the http port to tcp

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/instance: observability
  name: loki-memberlist
  namespace: observability
spec:
  clusterIP: None
  ports:
    - name: http
      port: 7946
      protocol: TCP
      targetPort: 7946
      appProtocol: tcp
  selector:
    app.kubernetes.io/instance: observability
    app.kubernetes.io/part-of: memberlist

Build from source

Wed, 06 Sep 2023 12:47:18 +0000

Build from source

Clone the Grafana Loki repository and use the provided Makefile to build Loki from source.

Prerequisites

Go, version 1.14 or later; set your $GOPATH environment variable
make
Docker (for updating protobuf and yacc files)

Build locally

Clone Loki to $GOPATH/src/github.com/grafana/loki:

git clone https://github.com/grafana/loki $GOPATH/src/github.com/grafana/loki

With a current working directory of $GOPATH/src/github.com/grafana/loki:
Bash
```
make loki
```

The built executable will be in $GOPATH/src/github.com/grafana/loki/cmd/loki/loki.