With the Grafana 8.5 release, we introduced the concept of service accounts. Now with the Grafana 9.1 release, we’re making service accounts generally available. This is a project that came out of technical necessity, but it has given us the opportunity to reflect on API tokens and machine-to-machine interaction across Grafana Labs.
There’s supposed to be a video here, but for some reason there isn’t. Either we entered the id wrong (oops!), or Vimeo is down. If it’s the latter, we’d expect they’ll be back up and running soon. In the meantime, check out our blog!
What is a service account?
Service accounts are Grafana users — but they’re not human, they’re machines. Service accounts are used to run automated or compute workloads, for example counting the number of data sources in Grafana every day or provisioning alerts using Terraform. These are tasks running in scripts or services outside of Grafana that require interacting with Grafana’s APIs.
For automations to use this new concept, they must create a service account token (SAT) linked to an existing service account. This token then allows authenticating against Grafana’s API as the service account with bearer token authentication.
curl --request GET \ --url http://mygrafana.example.orb/api/datasources \ --header 'Authorization: Bearer glpl_EX7x97tgxct383QhbrPIZgqPi9Q56w4H_7552804e'
In tandem with making role-based access control (RBAC) GA in Grafana 9.0 and expanding RBAC capabilities in Grafana 9.1, we want to make managing machine-to-machine interaction more secure while keeping it simple to use. We believe the following features further this goal.
Service account features
Link to an organization, not a user
A service account is linked to an organization, meaning that several administrators within the organization can manage the same service account. This removes the need of rotating tokens when an administrator leaves the organization.
Management permissions can also be restricted to specific administrators and transferred between users by using RBAC in Grafana Enterprise or Grafana Cloud Advanced.
Reduced permission sprawl
Service accounts have their own roles and permissions, allowing you to tailor the access of a service account and its tokens to the minimal set of permissions required. In conjunction with RBAC, you can grant the required permission for your external service, thereby reducing the risk of compromising your full instance in case of leaking a token or undue access to the external service.
Migrate existing API keys to a service account
The current system of API keys does not support the assignment of access control roles. You’ll have the option to migrate these keys to their own service accounts without requiring tokens to be rotated.
Multiple tokens associated with a single service account
Service accounts can have multiple tokens. It is easier to rotate tokens by using overlapping expiration times, and you can create separate tokens for each machine in a service. You can maintain the same set of permissions across these tokens and add new permissions if you realize they are missing.
Easier to identify tokens lurking in your code base
The current API key system in Grafana uses base64 encoded tokens, which can be mistaken for legitimate content in a code base. These tokens also shared the same format as Grafana Cloud tokens making it cumbersome to verify which Grafana stack component emitted said token.
Previous key format
New key format
We’ve changed service account tokens so they follow a visually identifiable pattern, thereby helping developers recognize when a token is hardcoded in the codebase. You’ll be able to identify service account tokens in specific tokens with a “glsa” prefix.
We’ve also added a checksum to the tokens for easier verification of token validity. We also hope this will help open source secret scanners reduce their rate of false positives and help users discover leaked keys.
Despite having a shorter length, this key format has the same entropy as the previous key format, ensuring we have not reduced the ease of brute-forcing a valid token.
What’s next for service accounts in Grafana?
The Grafana AuthNZ team is committed to continue delivering features that make authentication and authorization more complete and secure in Grafana.
If you want to find more information about service accounts or want to start using them today, you can check out the new service accounts documentation.
If you’re not already using Grafana Cloud — the easiest way to get started with observability — sign up now for a free 14-day trial of Grafana Cloud Pro, with unlimited metrics, logs, traces, and users, long-term retention, and access to one Enterprise plugin.