Deploy on Linux

This guide provides a step by step process for installing Grafana Enterprise Logs (GEL) on a Linux machine. It assumes you have access to a Linux machine and the permissions required to deploy a service with network and filesystem access. At the end of this guide you will have deployed a single GEL instance on a single node.

Prerequisites

You will need:

A valid Grafana Labs license with an associated GEL cluster name.
An s3 compatible object store

Setup an object storage bucket

GEL uses object storage as the backend for its logs storage (both the index and the log content). It also uses object storage for storing various administrative credentials and data related to the state of the system. (GEL and Loki support using the local filesystem as the backend for logs storage as well, but as this is not recommended for production deployments and is not supported for storing admin credentials, this guide will focus on setup with an object storage.)

This guide assumes that you are using Amazon S3 on the AWS us-east-1 region as your object store. If you plan on using a different region or object storage service, update the storage fields in the configuration file below. Currently, the supported object storage backends are AWS S3, other s3 compliant object stores, Google GCS, or Microsoft Azure’s object storage.

After you have provisioned an object storage backend, be sure to pre-create two buckets: grafana-logs-admin and grafana-logs-data. Those buckets will be referenced in the configuration file of this guide.

Install GEL

Run the following commands on every node as the root user:

# Add dedicated user and group
groupadd --system enterprise-logs
useradd --system --home-dir /var/lib/enterprise-logs -g enterprise-logs enterprise-logs

# Create directories
mkdir -p /etc/enterprise-logs /var/lib/enterprise-logs /var/lib/enterprise-logs/rules-temp
chown root:enterprise-logs /etc/enterprise-logs
chown enterprise-logs:enterprise-logs /var/lib/enterprise-logs /var/lib/enterprise-logs/rules-temp
chmod 0750 /etc/enterprise-logs /var/lib/enterprise-logs

# Download enterprise-logs binary
curl -Lo /usr/local/bin/enterprise-logs \
  https://dl.grafana.com/gel/releases/enterprise-logs-v1.7.2-linux-amd64
echo "f21e8b13d28235c4f4ffb552ac6cfae4203f488543dc56d95fb00169caf7a2c7" \
 "/usr/local/bin/enterprise-logs" | sha256sum -c
chmod 0755 /usr/local/bin/enterprise-logs
setcap 'cap_net_bind_service=+ep' /usr/local/bin/enterprise-logs

# Set up systemd unit and enable startup on boot
cat > /etc/systemd/system/enterprise-logs.service <<EOF
[Unit]
After=network.target

[Service]
User=enterprise-logs
Group=enterprise-logs
WorkingDirectory=/var/lib/enterprise-logs
ExecStart=/usr/local/bin/enterprise-logs \
   -config.file=/etc/enterprise-logs/enterprise-logs.yaml \
   -log.level=warn \

[Install]
WantedBy=default.target
EOF

systemctl daemon-reload
systemctl enable enterprise-logs.service

Create a GEL configuration file

Copy the following YAML config to a file called enterprise-logs.yaml. Update the cluster_name field with the name of the cluster your license was issued for and paste in your s3 credentials for the admin_client.

auth_enabled: true
auth:
  type: enterprise

server:
  http_listen_port: 3100

# TODO: Ensure the cluster name is set to match your Grafana Labs License file
cluster_name: # TODO: enter your cluster name

license:
  path: /etc/enterprise-logs/license.jwt

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 1h       # Any chunk not receiving new logs in this time will be flushed
  max_chunk_age: 1h           # All chunks will be flushed when they hit this age, default is 1h
  chunk_target_size: 1048576  # Loki will attempt to build chunks up to 1.5MB, flushing first if chunk_idle_period or max_chunk_age is reached first
  chunk_retain_period: 30s    # Must be greater than index read cache TTL if using an index cache (Default index read cache TTL is 5m)
  max_transfer_retries: 0     # Chunk transfers disabled

schema_config:
  configs:
    - from: 2020-10-24
      store: boltdb-shipper
      object_store: aws
      schema: v11
      index:
        prefix: index_
        period: 24h

storage_config:
  boltdb_shipper:
    active_index_directory: /var/lib/enterprise-logs/boltdb-shipper-active
    cache_location: /var/lib/enterprise-logs/boltdb-shipper-cache
    cache_ttl: 24h         # Can be increased for faster performance over longer query periods, uses more disk space
    shared_store: s3
  aws:
    region: us-east-1
    bucketnames: grafana-logs-data
    access_key_id: # TODO: insert your key id
    secret_access_key: # TODO: insert your secret key

limits_config:
  reject_old_samples: true
  reject_old_samples_max_age: 168h

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: false
  retention_period: 0s

ruler:
  storage:
    type: local
    local:
      directory: /var/lib/enterprise-logs/rules
  rule_path: /var/lib/enterprise-logs/rules-temp
  alertmanager_url: http://localhost:9093
  ring:
    kvstore:
      store: inmemory
  enable_api: true

compactor:
  working_directory: /var/lib/enterprise-logs/compactor
  shared_store: s3

admin_client:
  storage:
    s3:
      endpoint: s3.us-east-1.amazonaws.com
      bucket_name: grafana-logs-admin
      access_key_id: # TODO: insert your key id
      secret_access_key: # TODO: insert your secret key
    type: s3

Move the configuration file and license to the proper directory

Copy the configuration and the license files to all nodes in the GEL cluster:

enterprise-logs.yaml should be copied to /etc/enterprise-logs/enterprise-logs.yaml
license.jwt should be copied to /etc/enterprise-logs/license.jwt

Generate an admin token

Generate an admin token by running the following on a single node in the cluster:

# su enterprise-logs -c "/usr/local/bin/enterprise-logs \
   --config.file=/etc/enterprise-logs/enterprise-logs.yaml \
   --bootstrap.license.path=/etc/enterprise-logs/license.jwt \
   --log.level=warn \
   --target=tokengen"
Token created:  YWRtaW4tcG9saWN5LWJvb3RzdHJhcC10b2tlbjo8Ujc1IzQyfXBfMjd7fDIwMDRdYVxgeXw=

Start the enterprise-logs service

# systemctl start enterprise-logs.service

Verify your cluster is working

To verify your cluster is working, you can run the following command using the token you generated in the previous step.

curl -u :$API_TOKEN localhost:3100/ready

After running the above command, you should see the following output:

ready

This indicates the ingester component is ready to receive log data.

Next steps

To integrate your logs cluster with Grafana and a UI to interact with the Admin API, refer to Set up the GEL plugin for Grafana.