Deploy on Linux
This guide provides a step-by-step process for installing Grafana Enterprise Traces (GET) on Linux. It assumes you have access to a Linux machine and the permissions required to deploy a service with network and filesystem access. At the end of this guide, you will have deployed a single GET instance on a single node.
Before you begin
To follow this guide, you need:
- A running Grafana Enterprise instance (see installation instructions)
- A valid GET license with an associated GET cluster name
- An Amazon S3 compatible object store
System prerequisites
This configuration is an example you can use as a starting point. You may need to have more resources for your system than the minimum specifications listed below. Additional adjustments will be necessary for a production environment.
You must have the permissions required to deploy a service with a network and file system access.
Your Linux system should have at least:
- 4 CPUs
- 16 GB of memory
Setup an object storage bucket
GET uses object storage as the backend for its trace storage. It also uses object storage for storing various administrative credentials and data related to the state of the system.
GET and Tempo support using the local filesystem as the backend for trace storage as well. This is not recommended for production deployments and is not supported for storing admin credentials, this guide focuses on setup with an object storage.
This example uses Amazon S3 on the AWS us-east-1
region as your object store.
If you plan on using a different region or object storage service, update the storage fields in the configuration file below. Currently, the supported object storage backends are AWS S3, other S3-compliant object stores, and Google Cloud’s GCS.
After you have provisioned an object storage backend, create two buckets: grafana-traces-admin
and grafana-traces-data
.
These buckets will be referenced in the configuration file of this guide.
You may need to alter the bucket names to be globally unique.
Consider adding a prefix for your organization to the bucket, for example, myorg-grafana-traces-admin
and myorg-grafana-traces-data
, and then replacing the names in the rest of these instructions with those bucket names.
Install GET
For more installation options, visit the GET release page.
For a linux-amd64 installation, run the following commands via the command line interface on your Linux machine.
You need administrator privileges to do this by running as the root
user or via sudo
as a user with permissions to do so.
Add a dedicated user and group and then change the password for the user to
enterprise-traces
:groupadd --system enterprise-traces useradd --system --home-dir /var/lib/enterprise-traces -g enterprise-traces enterprise-traces yes enterprise-traces | passwd enterprise-traces
Create directories and assign ownership.
mkdir -p /etc/enterprise-traces /var/lib/enterprise-traces /var/lib/enterprise-traces/rules-temp /var/lib/enterprise-traces/wal/search chown root:enterprise-traces /etc/enterprise-traces chown enterprise-traces:enterprise-traces /var/lib/enterprise-traces /var/lib/enterprise-traces/rules-temp /var/lib enterprise-traces/wal /var/lib/enterprise-traces/wal/search chmod 0750 /etc/enterprise-traces /var/lib/enterprise-traces /var/lib/enterprise-traces/wal /var/lib/enterprise-traces/wal/search
Download the enterprise-traces binary, verify checksums, and add network capabilities to the binary:
curl -Lo /usr/local/bin/enterprise-traces \ https://dl.grafana.com/get/releases/enterprise-traces-v1.3.0-linux-amd64 echo d950922d2038c84620ebe63a21786b9eaf8d4ed5f1801e6664133520407e5e86 \ /usr/local/bin/enterprise-traces | sha256sum -c chmod 0755 /usr/local/bin/enterprise-traces setcap 'cap_net_bind_service=+ep' /usr/local/bin/enterprise-traces
Set up systemd unit and enable startup on boot:
cat > /etc/systemd/system/enterprise-traces.service <<EOF [Unit] After=network.target [Service] User=enterprise-traces Group=enterprise-traces WorkingDirectory=/var/lib/enterprise-traces ExecStart=/usr/local/bin/enterprise-traces \ -config.file=/etc/enterprise-traces/enterprise-traces.yaml \ -log.level=warn \ [Install] WantedBy=default.target EOF systemctl daemon-reload systemctl enable enterprise-traces.service
Create a GET configuration file
Copy the following YAML configuration to a file called enterprise-traces.yaml
.
Paste in your S3 credentials for admin_client and the storage backend. If you wish to give your cluster a unique name, add a cluster property with the appropriate name. If you do not add a cluster name this will be taken automatically from the license.
By default, the cluster_name
Update the cluster_name
field with the name of the cluster your license was issued for and paste in your S3 credentials for the admin_client
.
Refer to the GET configuration documentation for explanations of the available options.
In the following configuration, GET options are altered to only listen to the OTLP gRPC and HTTP protocols. By default, GET listens for all compatible protocols. The extended instructions for installing the TNS application and Grafana Agent to verify that GET is receiving traces relies on the default Jaeger port being available, hence disabling listening on that port in GET for a single Linux node.
multitenancy_enabled: true
auth:
type: enterprise
license:
path: /etc/enterprise-traces/license.jwt
http_api_prefix: /tempo
distributor:
receivers:
otlp:
protocols:
grpc:
http:
ingester:
lifecycler:
ring:
replication_factor: 3
server:
http_listen_port: 3200
storage:
trace:
backend: s3
s3:
endpoint: s3.us-east-1.amazonaws.com
bucket: grafana-traces-data
forcepathstyle: true
#set to true if endpoint is https
insecure: true
access_key: # TODO: insert your key id
secret_key: # TODO: insert your secret key
wal:
path: /var/lib/enterprise-traces/wal
admin_api:
leader_election:
enabled: false
admin_client:
storage:
s3:
endpoint: s3.us-east-1.amazonaws.com
bucket_name: grafana-traces-admin
access_key_id: # TODO: insert your key id
secret_access_key: # TODO: insert your secret key
type: s3
Move the configuration file and license to the proper directory
The enterprise-traces.yaml and license.jwt files need to be moved:
enterprise-traces.yaml
should be copied to/etc/enterprise-traces/enterprise-traces.yaml
license.jwt
should be copied to/etc/enterprise-traces/license.jwt
Copy the configuration and the license files to all nodes in the GET cluster:
cp enterprise-traces.yaml /etc/enterprise-traces/enterprise-traces.yaml
cp license.jwt /etc/enterprise-traces/license.jwt
Generate an admin token
Generate an admin token by running the following on a single node in the cluster, using the password for the
enterprise-traces
user set earlier:su enterprise-traces -c "/usr/local/bin/enterprise-traces \ --config.file=/etc/enterprise-traces/enterprise-traces.yaml \ --license.path=/etc/enterprise-traces/license.jwt \ --log.level=warn \ --target=tokengen" # Token created: YWRtaW4tcG9saWN5LWJvb3RzdHJhcC10b2tlbjo8Ujc1IzQyfXBfMjd7fDIwMDRdYVxgeXw=
After you enter your password, the system outputs a new token. Save this token somewhere secure for future API calls and to enable the GET plugin.
Password: Token created: YourTokenHere12345
You can export the API token to use later in the procedure using the command below. Replace the value for the token with the one you generated.
export API_TOKEN=YourTokenHere12345
Start the enterprise-traces service
Use systemctl
to start the service:
systemctl start enterprise-traces.service
You can replace start
with stop
to stop the service.
Verify your cluster is working
To verify your cluster is working, run the following command using the token you generated in the previous step.
curl -u :$API_TOKEN localhost:3200/ready
After running the above command, you should see the following output within 30-60 seconds:
ready
This indicates the ingester component is ready to receive trace data.
Set up the GET plugin
Refer to Set up the GET plugin for Grafana to integrate your GET cluster with Grafana and a UI to interact with the Admin API.