Documentation Grafana Cloud Instrument and send data Logs Cloud Logs Export Validate Cloud Logs Export Data

Validate Cloud Logs Export Data

Cloud Logs Export validates your data before making it available, but it can also be inspected and validated by you through the lokitool command line interface.

To validate you have to know which index you’d like to validate. In case you don’t know where to start:

1. In Grafana Cloud, select Explore from the main menu.
2. On the Explore page, set the data source to the one that starts with insightlogs.
3. Search for index successfully audited in the logs. You’ll see log lines that contains an index field.
4. Pick one of the listed indexes. This is the index you’ll validate in the next steps. For this example the selected index is index/my_loki_index_19856/12345/1715707992713992005-compactor-1715189977885-1715707796275-f8003361.tsdb.gz.

With the desired index in hand, you now:

1. Download the lokitool from the Releases page that matches your Loki version.

2. Create a YAML configuration with the structure shown below. The schema_config and storage_config can be the same you have used previously to query CLE data. In this example this file is named cle-config.yaml.

   yaml Copy

   schema_config:
     configs:
       - from: '2023-08-21'
         index:
           period: 24h
           prefix: my_index_
         object_store: gcs
         schema: v13
         store: tsdb
   
   storage_config:
     gcs:
       bucket_name: my-loki-bucket
   
   tenant: <your Grafana Cloud Logs tenant>

3. Run the downloaded lokitool binary the following way:

   sh Copy

   ./lokitool-linux-amd64 audit index  --config.file=deprecated
     --index.file=index/my_loki_index_19856/12345/1715707992713992005-compactor-1715189977885-1715707796275-f8003361.tsdb.gz
     --  --period=19856 --config.file=cle-config.yaml

   • The --period configuration represents the period of the index being audited. You can find it by checking the 5-digits number appended as a suffix of the Loki environment name in the index file. For example: For index/loki_env_tsdb_index_19856/12345/..., the period is 19856.
   • The --config.file is the YAML configuration file created by you in the second step. Notice that the first appearance of this configuration is before the isolated -- symbol and is deprecated (to be removed on the next major); what really counts is the second appearance, after the isolated -- symbol.
   • The --index.file is the path to the index file you want to audit. You can look at your bucket to see the exactly path of an index you’d like to validate.

4. Review the results. If any error shows up make sure to report it to Grafana Cloud Logs support.