Delete data from your exported log archive

Cloud Logs Export writes a synced copy of your logs into an object storage bucket that you own and manage. Because you own the bucket, you’re responsible for deleting or otherwise managing objects in it. This topic explains how the exported data is structured, so you can target the right objects, and clarifies how deleting log lines from Grafana Cloud Logs relates to the copy already written to your bucket.

How your exported logs are structured

Cloud Logs Export writes your logs to your bucket using the Loki open source chunk and index formats. Refer to Cloud Logs Export and Query your exported logs for more on these formats.

Objects in your bucket fall into these categories:

Chunk objects hold the compressed log content, under a path in the form <TENANT_ID>/<CHUNK_ID>/<CHUNK_FILE>. These hold the log lines you want to remove.
Index objects map label sets to the chunks that contain them, under a path in the form index/<INDEX_PREFIX>_<TABLE_NUMBER>/<TENANT_ID>/<FILE>.tsdb.gz. The export includes the compacted index for each day, and TABLE_NUMBER counts days since the Unix epoch. For an example of a real index path, refer to Validate Cloud Logs Export Data.
Schema config files are written at the root of your bucket as <TENANT_ID>-schemaconfig.yaml. Tools like LogCLI use them to read the index and chunk objects together, as described in Query your exported logs. When your schema changes, Cloud Logs Export adds a numbered file, for example <TENANT_ID>-schemaconfig-1.yaml, instead of overwriting the existing one.

Every path includes your Grafana Cloud Logs tenant ID. To find your tenant ID:

In the Cloud Portal, click Details on the tile for your stack.
Click Details on the Loki tile.
Under Grafana Data Source Settings, copy the value from the User field.

INDEX_PREFIX comes from your schema configuration, so confirm it against your own bucket before you delete anything or write a lifecycle rule. Validate Cloud Logs Export Data describes how to locate the path of a specific object in your bucket.

If you configured per-stream export filters, they only change what’s exported going forward. They don’t change the structure of, or remove, data that was already written to your bucket before the filter’s startDate took effect.

For the full technical specification of the chunk binary layout and the object types Loki writes to a bucket, refer to Chunk Format and Bucket object layout in the Loki documentation.

How deletion from Grafana Cloud Logs affects your exported archive

Delete unwanted information in log lines describes how to use the Loki delete API to remove log lines from Grafana Cloud Logs. That API acts on the data stored in Grafana Cloud Logs. Cloud Logs Export only writes to your bucket and never deletes from it, so a delete request never removes objects that Cloud Logs Export already wrote there.

To remove sensitive or unwanted data from both places, work in this order:

Delete the log lines from Grafana Cloud Logs using the Loki delete API. Deletion runs as a background task every 24 hours, so allow time for it to finish.
Confirm the log lines are gone from Grafana Cloud Logs, then delete the matching objects from your bucket.

The order matters, because Cloud Logs Export can copy an object again while the source data still exists.

Delete data that’s still within the export window

Cloud Logs Export syncs logs that are between 7 and 30 days old, as described in Cloud Logs Export. While a day falls in that window:

Cloud Logs Export skips objects that already exist in your bucket, but copies an object again if it’s missing or a different size. If you delete an object while the source log lines still exist in Grafana Cloud Logs, expect the object to reappear.
Cloud Logs Export checks that the chunks an exported index expects are present in your bucket. Deleting chunk objects that an index still refers to can delay further exports. If exports stop progressing after you delete objects, contact Support.

After a day passes out of the 30 day window, Cloud Logs Export no longer copies data for it, so deletions you make are permanent.

Delete objects from your cloud storage bucket

Grafana doesn’t provide an API or UI to delete objects from your export bucket. Because the bucket belongs to you, use your cloud provider’s own console, CLI, SDK, or lifecycle-policy tooling to delete objects or configure automatic expiration.

When you delete objects or set up a lifecycle rule:

Scope the rule to the chunk object prefix for your tenant. A rule that targets the whole bucket also removes the schema configuration and index objects that the archive needs to stay queryable.
Keep the <TENANT_ID>-schemaconfig.yaml files and any index objects that cover data you want to keep. Without them, you can’t query the rest of the archive with LogCLI or read-only Loki, as described in Query your exported logs.
Expect the index to keep referring to chunk objects that you delete. Queries that cover the same time range can fail as a result. Refer to Log retention in the Loki documentation for how Loki treats index references to chunks that no longer exist.
Select chunk objects by path, not by the timestamps of the logs inside them. Chunk object names don’t include the timestamps of the log lines they hold. To find the objects for a given day, use the index object for that day, which you can identify from its table number.
Cloud Logs Export supports write-once buckets and never overwrites a file after it writes it, as described in Cloud Logs Export. If you turned on write-once or immutability features for your bucket, check whether those settings restrict deletion.

For instructions on deleting objects or configuring lifecycle policies in your bucket, refer to your cloud provider’s documentation for the storage service you’re using.

Prevent recurrence

To reduce the chance of sensitive data reaching your export bucket again, refer to Per stream export to configure filters that limit which log streams Cloud Logs Export syncs.