Retain logs longer without breaking the bank: Introducing Grafana Cloud Logs Export
Late last year we announced an early access program for Grafana Cloud Logs Export, a feature that allows users to easily export logs from Grafana Cloud to their own cloud-based object storage for long-term archival purposes. We are pleased to announce that the feature is now in public preview for all Grafana Cloud users, including those on the Free tier!
With Cloud Logs Export, you can cost-effectively archive logs on the object storage — Amazon S3, Google Cloud Storage (GCS), Microsoft Azure Blob — of your choice. With this feature, which is cloud-only at this time, users can export log data at no additional cost. They can then access the exported data either with LogCLI or by using a self-hosted version of Grafana Loki. It’s a great fit if you have compliance or audit requirements that necessitate long-term log retention.
Why we built Cloud Logs Export
Many customers have told us they need to retain logs longer than 30 days, which is the default duration in Grafana Cloud Logs. Many of these use cases are compliance-related; customers simply need to retain logs for longer periods, even though they only access them infrequently to service audit requests. Other times, they need to search their logs to analyze past incidents that extend farther back than the standard observability use cases.
Today, Grafana Cloud can already address this by simply extending the retention period for logs longer than the standard 30 days. However, we heard two pieces of feedback that motivated us to build Cloud Logs Export:
Cost. While extended retention is a great fit for use cases that involve querying older data, many customers told us they would prefer a cheaper retention option; their underlying goal is to have the data on hand to query ad hoc as cheaply as possible.
Control. Other customers simply want better control over their log data. They’re facing compliance requirements that specify that they must directly control certain data; or they have concerns about being locked into Grafana Labs as a vendor; or they have a desire to perform downstream analytics or transformations on that data, which they can’t do while it’s in Grafana Cloud.
We realized that enabling customers to use their own object storage for archived logs — after all, they are already in the cloud! — would neatly solve both of these problems.
Configure long-term retention on your object storage of choice
Cloud Logs Export supports all three major cloud providers’ object storage solutions: S3, GCS, and Azure Blob Storage. As part of this public preview release, users will be able to configure this functionality entirely on their own through the Grafana UI.
To get started, simply follow these steps:
- Log into your Grafana Cloud account. If you don’t already have one, you can sign up for one for free today.
- From your homepage, click on Administration in the left-hand navigation, then select Plugins.
- In the search bar, start typing “Cloud Logs Exporter,” at which point you should see the appropriate plugin. Click on it.
- Select your target cloud provider, follow the appropriate configuration steps, then enter the appropriate details, including the object storage name and region.
- Test the connection, then select Submit.
There’s supposed to be a video here, but for some reason there isn’t. Either we entered the id wrong (oops!), or Vimeo is down. If it’s the latter, we’d expect they’ll be back up and running soon. In the meantime, check out our blog!
To ensure data is syncing as intended, users will also have access to a dashboard that shows the status of their sync, as well as understand what data has been exported.
Note: If “Cloud Logs Export” is not enabled for self-service in your region, Grafana Cloud Pro and Advanced tier users can request access by filing a support request.
Zero cost to query archived logs
Of course, customers also told us they would need to query this data, in addition to retaining it.
To ensure we could offer this feature in a no-cost manner to our customers, data is exported in a compressed, Loki “chunk” format. This means it can be queried in one of two ways:
- Using LogCLI
- Using a read-only self-hosted version of Loki
Try Grafana Cloud Logs Export today
Available to all Grafana Cloud users and tiers, you can take this feature for a spin using the Cloud Logs Exporter plugin. We are excited that this will unlock use cases for cost-effective long-term log storage. We’d love to hear what you think!
Grafana Cloud is the easiest way to get started with metrics, logs, traces, and dashboards. We have a generous forever-free tier and plans for every use case. Sign up for free now!
