The admin API allows you to create and manage resources in your Grafana Enterprise Traces (GET) cluster such as tenants, access policies, and tokens. When performing operations that modify resources such as updates or deletes, the admin API requires the use of resource version numbers to make sure the most recent version of resources are being operated on. An additional, optional, safety feature the admin API can make use of to ensure concurrent modifications don’t overwrite each other is leadership election. This allows admin API components to elect a single component to perform all modification requests, making sure that simultaneous requests don’t overwrite each other. Details about how to use and configure this feature are provided below.
Leadership election is an optional, but recommended, feature of the admin API. It can be enabled and
configured under the
admin_api section of the GET configuration file.
Leadership election uses the same mechanism as other GET components to coordinate among themselves, the
hash ring. This means you’ll need to configure
this instance of the ring the same way you configured the ring used by distributors and ingesters when
setting up your GET cluster.
An example of this configuration is provided below using the memberlist gossip ring.
admin_api: leader_election: enabled: true ring: kvstore: store: memberlist tokens_observe_period: 5s
Note that any supported backend (Etcd, Consul, or memberlist) can be used for leader election with the admin API; memberlist is only used as an example here. For simplicity, it is recommended to use whatever ring backend you are already making use of for other GET components.
Admin API leadership election does not require a quorum to operate like other uses of the ring in GET components. This means it is not required to run an odd number of admin API components. Instead, it is sufficient to run two for redundancy.
Related Enterprise Traces resources
Get started with distributed tracing with Grafana Tempo and Grafana Enterprise Traces
In this ObservabilityCON session, find out the latest features in the open source tracing project and the new addition to Grafana Enterprise Stack.
Correlate your metrics, logs, and traces with Grafana Stack
Show how Grafana’s opinionated stack will allow you to correlate your metrics, logs, and traces, all under one roof.