Migrate to TSDB

TSDB is the recommended index type for Loki and is where the current development lies. If you are running Loki with boltb-shipper or any of the legacy index types that have been deprecated, we strongly recommend migrating to TSDB.

Configure TSDB index for an upcoming period

To begin the migration, add a new period_config entry in your schema_config. You can read more about schema config here.

Note
You must roll out the new period_config change to all Loki components in order for it to take effect.

This example adds a new period_config which configures Loki to start using the TSDB index for the data ingested starting from 2023-10-20.

schema_config:
  configs:
    - from: 2023-01-01
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h
    - from: 2023-10-20 ①
      store: tsdb ②
      object_store: filesystem ③
      schema: v13 ④
      index:
        prefix: index_
        period: 24h

You must set the new period from to a date in the future.
Update the new period to use TSDB as the index type by setting store: tsdb.
This sample configuration uses filesystem as the storage in both the periods. If you want to use a different storage for the TSDB index and chunks, you can specify a different object_store in the new period.
Update the schema to v13 which is the recommended version at the time of writing. Please refer to the configure page for the current recommended version.

Configure TSDB shipper

It’s also important that you configure the tsdb_shipper block in storage_config. Specifically the following options:

active_index_directory: directory where ingesters would write index files which will then be uploaded by shipper to configured storage.
cache_location: cache location for downloading index files from the storage for use in query path.

storage_config:
  tsdb_shipper:
    active_index_directory: /data/tsdb-index
    cache_location: /data/tsdb-cache

Run compactor

We strongly recommended running the compactor when using TSDB index. It is responsible for running compaction and retention on TSDB index. Not running index compaction will result in sub-optimal query performance.

Please refer to the compactor section for more information and configuration examples.