As an application,
Loki is built out of many component microservices,
and is designed to run as a horizontally-scalable distributed system.
Loki’s unique design compiles the code of the entire distributed system into
a single binary or Docker image.
The behavior of that single binary is controlled with
-target command-line flag and defines one of three modes of operation.
Configuration for each deployed instance of the binary further specifies which components it runs.
Loki is designed to easily redeploy a cluster under a different mode as your needs change, with no configuration changes or minimal configuration changes.
The simplest mode of operation sets
This is the default target,
and it does not need to be specified.
This is monolithic mode;
it runs all of Loki’s microservice components inside a single process
as a single binary or Docker image.
Monolithic mode is useful for getting started quickly to experiment with Loki, as well as for small read/write volumes of up to approximately 100GB per day.
Horizontally scale up a monolithic mode deployment to more instances
by using a shared object store, and by configuring the
to share state between all instances.
High availability can be configured by running two Loki instances
memberlist_config configuration and a shared object store.
Route traffic to all the Loki instances in a round robin fashion.
Query parallelization is limited to the quantity of instances and the query parallelism defined.
Simple scalable deployment mode
If your volume of logs exceeds a few hundred GB a day, or if you would like to separate read and write concerns, Loki provides the simple scalable deployment mode. This deployment mode can scale to several TBs of logs per day and more. Consider the microservices mode approach for very large Loki installations.
In this mode the component microservices of Loki are bundled into two targets:
The BoltDB compactor
service will run as part of the read target.
There are advantages to separating the read and write paths:
- higher availability of write path by providing dedicated nodes
- separately scalable read path to add/remove query performance on demand
The simple scalable deployment mode requires a load balancer in front of Loki,
/loki/api/v1/push traffic to the write nodes.
All other requests go to the read nodes.
Traffic should be sent in a round robin fashion.
The simple scalable deployment of Loki can scale to several TBs of logs per day and more.
The microservices deployment mode instantiates components of Loki
as distinct processes.
Each process is invoked specifying its
Running components as individual microservices allows scaling up by increasing the quantity of microservices. The customized cluster has better observability of the individual components. Microservices mode deployments are the most efficient Loki installations. However, they are also the most complex to set up and maintain.
Microservices mode is recommended for very large Loki clusters or for clusters that require more control over scaling and cluster operations.
Microservices mode works best with Kubernetes deployments. There are Jsonnet and distributed Helm chart installations.
Related Enterprise Logs resources
Grafana Enterprise Logs: Logging with security and scale
Join us for this webinar, which will cover: Challenges with logging as organizations scale and the volume of logs explodes, how Grafana Enterprise Logs enables organizations to make logs available to any team members who need them, features available in GEL and how to get access, a live product demo so you can see GEL for the first time
VIDEO: Watch this first-look demo of the new Grafana Enterprise Logs
Based on Loki, Grafana Enterprise Logs is part of the Grafana Enterprise Stack for composing and scaling observability on your own infrastructure.
Introducing Grafana Enterprise Logs, a core part of the Grafana Enterprise Stack integrated observability solution
Powered by the Loki open source project, the Enterprise Logs offering joins metrics and dashboards in our enterprise-ready stack for self-managed observability.