This is documentation for the next version of Metrics enterprise. For the latest stable release, go to the latest version.

Set up GEMDeploy on Linux

Deploy GEM on a Linux host

This guide provides a step by step process for installing Grafana Enterprise Metrics on a Linux machine. It assumes you have access to a Linux machine and the permissions required to deploy a service with network and filesystem access. At the end of this guide you will have deployed a single GEM instance on a single node.

Prerequisites

In order to follow this guide you will the following:

  • A valid Grafana Labs license with an associated GEM cluster name.

Setup an object storage bucket

To begin, you need access to an object storage backend. GEM uses object storage as a backend to store time series data as well as data related to the state of the system. This topic assumes that you are using Amazon S3 on the AWS us-east-1 region. If you plan on using a different region or object storage service, update the storage fields in the configuration file to match below to configure your desired service. Currently, the supported services are object storage backends that support the S3 API or Google GCS.

After you have provisioned an object storage backend, be sure to pre-create two buckets: grafana-metrics-admin and grafana-metrics-tsdb. Those buckets will be referenced in the configuration file of this guide.

Prepare your system

Before running GEM, it is recommended you setup a system user and prepare the required directories on the filesystem. Run the following commands on every node as the root user. To login as the root user run:

sudo su -

Add dedicated user and group

groupadd --system ge-metrics
useradd --system --home-dir /var/lib/ge-metrics -g ge-metrics ge-metrics

Create directories and assign ownership

mkdir -p /etc/ge-metrics /var/lib/ge-metrics
chown ge-metrics:ge-metrics /etc/ge-metrics
chown ge-metrics:ge-metrics /var/lib/ge-metrics
chmod 0750 /etc/ge-metrics /var/lib/ge-metrics

Copy the license file

You will need to ensure you license token file is copied to the path /etc/ge-metrics/license.jwt.

Create a GEM configuration file

Copy the following YAML config to a file called /etc/ge-metrics/config.yaml.

auth:
  type: enterprise

target: all

license:
  path: /etc/ge-metrics/license.jwt

admin_client:
  storage:
    type: s3
    s3:
      endpoint: s3.amazonaws.com
      bucket_name: grafana-metrics-admin
      access_key_id: # TODO: insert access key id here
      secret_access_key: # TODO: insert secret access key here
distributor:
  shard_by_all_labels: true
  pool:
    health_check_ingesters: true

memberlist:
  abort_if_cluster_join_fails: false
  bind_port: 7946

ingester:
  lifecycler:
    num_tokens: 512
    ring:
      kvstore:
        store: inmemory
      replication_factor: 1

blocks_storage:
  tsdb:
    dir: /tmp/cortex/tsdb
  bucket_store:
    sync_dir: /tmp/cortex/tsdb-sync
  backend: s3
  s3:
    endpoint: s3.amazonaws.com
    bucket_name: grafana-metrics-tsdb
    access_key_id: # TODO: insert access key id here
    secret_access_key: # TODO: insert secret access key here

storage:
  engine: blocks

Next you will need to update the config file and set the blocks_storage.s3 and admin_client.storage.s3 sections to include access credentials for your object storage backend.

Download and configure the GEM binary

curl -Lo /usr/local/bin/ge-metrics https://dl.grafana.com/gem/releases/metrics-enterprise-v1.6.1-linux-amd64
echo "64f307e1f6f72cb8557c6d2ef0c8eb2aa939ddfa358bf53c2705fb50e4c0f253 /usr/local/bin/ge-metrics" | sha256sum -c
chmod 0755 /usr/local/bin/ge-metrics
setcap 'cap_net_bind_service=+ep' /usr/local/bin/ge-metrics

Set up systemd unit

Copy the following file to the path /etc/systemd/system/ge-metrics.service:

[Unit]
After=network.target

[Service]
User=ge-metrics
Group=ge-metrics
WorkingDirectory=/var/lib/ge-metrics
ExecStart=/usr/local/bin/ge-metrics \
   -config.file=/etc/ge-metrics/config.yaml

[Install]
WantedBy=default.target

Enable startup at boot timeand start Grafana Enterprise Metrics

systemctl daemon-reload
systemctl enable ge-metrics.service
systemctl start ge-metrics.service

Generate an admin token

Generate an admin token by running the following on a single node in the cluster:

su ge-metrics -c "/usr/local/bin/ge-metrics \
--config.file=/etc/ge-metrics/config.yaml \
--target=tokengen"

The output of the above command should contain a token:

Token created:  YWRtaW4tcG9saWN5LWJvb3RzdHJhcC10b2tlbjo8Ujc1IzQyfXBfMjd7fDIwMDRdYVxgeXw=

Verify your cluster is working

To verify your cluster is working you can run the following command using the token you generated in the previous step. For example:

curl -u :YWRtaW4tcG9saWN5LWJvb3RzdHJhcC10b2tlbjo8Ujc1IzQyfXBfMjd7fDIwMDRdYVxgeXw= localhost/ready

After running the above command you should see the following output:

ready

Next steps

Now that you have a working GEM deployment locally, refer to setup up the Grafana Enterprise Metrics plugin for instructions on how to integrate your metrics cluster with Grafana and give you a UI to interact with the Admin API.