Pyroscope Block format

This document describes how Pyroscope stores the data in its blocks. Each block belongs to a single tenant and is identified by a unique ULID. Within the block there are multiple files:

A metadata file meta.json, which contains information about what the block contains, like the time range of the profiling data.
A TSDB index index.tsdb mapping the external labels to the profiles stored in the profiles table.
profiles.parquet parquet table that contains profiles.
symbols sub-directory contains profiling symbols that provide a link between the compiled or interpreted binary code and the original source code:
- A index.symdb file with meta information, which helps to find symbols for a specific profile.
- A stacktraces.symdb file contains stack traces compacted in the parent pointer tree.
- Parquet tables for models referenced by stack traces: locations.parquet, functions.parquet, mappings.parquet, strings.parquet.

Data model

The data model within the block is fairly aligned to Google’s proto definition for the pprof wire format.

Profile series labels contain additional information gathered at ingestion time and can be used to select certain profiles. They are comparable to Prometheus/Loki labels and typical label names are namespace and pod to describe which workload the profiles are coming from.

Each profile ingested will be added into a new row in the profile table. If there are entries missing in the tables for the different models they are also inserted.