Secure credential storage for your observability stack: Introducing secrets management in Grafana Cloud

The more your infrastructure grows, the more likely you are to face a familiar challenge: where to safely store the API keys, passwords, and tokens that power your observability stack. 

Unfortunately, a common response to this dilemma is to scatter credentials across configurations, making security and management of secrets increasingly complex. The problem is only compounded by modern observability setups that involve distributed services, API integrations, and collaborative teams—all requiring secure access to sensitive information.

That’s why we’re launching secrets management in Grafana Cloud. Currently in public preview, it’s designed to be the foundation for securing all your credentials in Grafana. And we aren’t just adding another feature with secrets management; we’re solving the credential management problem that every growing team faces by delivering these key benefits:

• Centralized credential storage with enterprise-grade encryption
• Reference-based secrets access 
• Granular access controls
• Complete audit trail of secret usage

In this blog, we’ll show you how we’re starting this rollout with Grafana Cloud Synthetic Monitoring and how you can implement it today. We’ll also give you a look behind the scenes to show how we’re developing this new functionality.

Secrets management in action: public preview for Synthetic Monitoring

The first product in Grafana Cloud to integrate with our new secrets manager is Synthetic Monitoring. This has been a highly requested feature that delivers clear and immediate value for users, so let’s look at how this changes what you can do in Synthetic Monitoring.  

The old way: embedded credentials   

If you’re running Synthetic Monitoring checks, you’ve likely faced this scenario:

Copy

const bearerToken = 'abc123-hardcoded-token-visible-to-everyone'; 
const response = http.get('https://api.example.com/data', { 
  headers: { 'Authorization': `Bearer ${bearerToken}` } 
});

There are several problems with this approach:

• Credentials are visible in the checks.
• Team collaboration requires sharing sensitive values.
• There’s no audit trail of credential usage.

The new way: secret references

Copy

const bearerToken = await secrets.get('my-token'); 
const response = http.get('https://api.example.com/data', { 
  headers: { 'Authorization': `Bearer ${bearerToken}` } 
});

With secrets management in Grafana Cloud, a user can store values in their stack, then anyone can configure a check that references those values by name (e.g., in an API request header). This was inspired by how GitHub handles secrets in CI actions. 

There are several benefits to this approach:

• Sensitive values are stored securely and separately.
• Multiple checks can reference the same secret.
• You get easy credential rotation without touching individual checks.
• There’s a full audit trail of which service accessed what and when.

Try it today

As we mentioned, secrets management is available now in public preview for all Grafana Cloud customers. Here’s how to get started:

1. Access the feature: Log into your Grafana Cloud instance and navigate to Testing & synthetics > Synthetics > Config
2. Create your first secret: Start with an API key or credential you’re currently using in Synthetic Monitoring
3. Add the new secrets module to your check: import secrets from ‘k6/secrets’; 
4. Update your checks: Replace hardcoded values with secret references using secrets.get('my-secret') syntax

You can see how the process works in the UI below. For more details, check out our documentation.

How we built this securely: the technical foundation

Building a secrets manager for Grafana Cloud required rethinking security from the ground up. Here’s how we designed it to protect your most sensitive data.

Encryption process

We encrypt secrets with envelope encryption so that they are protected at rest. Envelope encryption uses randomly generated data keys to encrypt secrets using an AES-GCM cipher, then encrypts those data keys with a root key before storing them. 

This obfuscates the data and spreads the decryption keys around, making successful attacks harder. Grafana Cloud supports envelope encryption using Key Management Services (KMS) in AWS, Google Cloud, and Microsoft Azure, as well as with Hashicorp Vault or a self-managed key. 

Our security principles

Encryption is just the beginning. We built secrets management in Grafana Cloud around core security principles that protect your data while making the system easier to use and manage.

Unified API and storage

In collaboration with our Application Security team, we designed a central secrets repository that maintains the highest security standards across all services. Services can use secrets without implementing their own envelope encryption or handling secret rotation and re-encryption. Secrets management in Grafana Cloud handles all of this automatically.

Metadata separation

Secret values are stored completely separately from metadata such as names and descriptions. This enables team collaboration and secret reuse without ever exposing sensitive data.

Secrets can only be decrypted by internal services

Users cannot access the sensitive payload. All services making decryption requests are rigorously vetted and validated for authenticity, permissions, and explicit access grants.

Adherence to encryption best practices

We continue to protect secrets with envelope encryption, including using popular third-party encryption services such as HashiCorp Vault and AWS Secrets Manager. Encryption keys are regularly rotated and all interactions are audited.

Observable

At Grafana Lab, it is a core part of our technical strategy to dogfood our own observability products. As such, secrets management in Grafana Cloud was instrumented for RED Method analysis using logs, traces, and metrics to detect any potential issues. 

Note: For more information on how we maintains high standards of data privacy and security, check out our security compliance page.

Going beyond Synthetic Monitoring

Synthetic Monitoring is just the beginning for secrets management in Grafana Cloud. We’re already working on exciting expansions.

• Third-party secret stores: Connect your existing HashiCorp Vault, AWS Secrets Manager, Google Cloud Secret Manager, or Azure Key Vault to Grafana Cloud.
• Data source credentials: Integrate data sources with secrets management in Grafana Cloud and reference secrets stored in your third‑party secret store.
• Grafana Cloud k6 integration: Reference secrets directly in your k6 scripts without embedding credentials in code.

Building trust through security

At Grafana Labs, we understand that secrets are at the core of trust between us and our users. With secrets management in Grafana Cloud, we’re not just solving today’s credential management challenges—we’re building the foundation for secure, scalable secrets management that will grow with your observability needs.

This represents a significant investment in security infrastructure that will enable new capabilities across the entire Grafana ecosystem. From Synthetic Monitoring to data sources to future integrations we haven’t even imagined yet, you’ll have a secure, auditable way to manage the credentials that power your observability stack.

Grafana Cloud is the easiest way to get started with metrics, logs, traces, dashboards, and more. We have a generous forever-free tier and plans for every use case. Sign up for free now!