DocumentationGrafana CloudInstrument and send dataLogsCloud Logs ExportValidate Cloud Logs Export Data

Validate Cloud Logs Export Data

Cloud Logs Export validates your data before it making available but it can also be inspected and validated by you, through the lokitool command line interface.

To validate you have to know which index you’d like to validate. In case you don’t know where to start:

1. In Grafana, access the Explore page and set the data source to the one that starts with insightlogs.
2. Search for index successfully auditted on the logs. You’ll see log lines that contains an index field.
3. Pick one of the listed indexes. This is the index you’ll validate in the next steps. For this example the picked index is index/my_loki_index_19856/12345/1715707992713992005-compactor-1715189977885-1715707796275-f8003361.tsdb.gz.

With the desired index in hand, you now:

1. Download lokitool on the Releases page.
2. Create an YAML configuration with the structure shown below. The schema_config and storage_config can be the same you have used previously to query CLE data. On this example this file is named cle-config.yaml.

yaml Copy

schema_config:
  configs:
    - from: '2023-08-21'
      index:
        period: 24h
        prefix: my_index_
      object_store: gcs
      schema: v13
      store: tsdb

storage_config:
  gcs:
    bucket_name: my-loki-bucket

tenant: <your Grafana Cloud Logs tenant>

1. Run the downloaded lokitool binary the following way:

sh Copy

./lokitool-linux-amd64 audit index  --config.file=deprecated
  --index.file=index/my_loki_index_19856/12345/1715707992713992005-compactor-1715189977885-1715707796275-f8003361.tsdb.gz
  --  --period=19856 --config.file=cle-config.yaml

• The --period configuration represents the period of the index being audited. You can find it by checking the 5-digits number appended as a suffix of the Loki environment name in the index file. Example: For index/loki_env_tsdb_index_19856/12345/..., the period is 19856.
• The --config.file is the YAML configuration file created by you in the second step. Notice that the first appearance of this configuration is before the isolated -- symbol and is deprecated (to be removed on the next major); what really counts is the second appearance, after the isolated -- symbol.
• The --index.file is the path to the index file you want to audit. You can look at your bucket to see the exactly path of an index you’d like to validate.

1. See the results. If any error shows up make sure to report to Grafana Cloud Logs support.