Before you begin
Follow the checklist to ensure that your application is generating metrics, traces, and exemplars.
- Verify that your application is using the official Prometheus client libraries.
- Ensure that the client library you choose is emitting metrics in OpenMetrics format by referencing its documentation. For the Prometheus Go client library, for example, this requires you to set
EnableOpenMetrics
totrue
. For the Java library, follow its instructions on setting the proper header format. - Obtain the trace ID for the current request and include the trace ID in calls to emit metrics.
- For histograms, use the
ObserveWithExemplar
method to emit the trace ID along with a value for the histogram. These functions are from the Go library but you can find similar functions in the other libraries. - For counters, use the
AddWithExemplar
method to emit the trace ID along with a counter increment.
- For histograms, use the
- Verify that metrics are being generated with exemplars by running the following command in a shell:
curl -H "Accept: application/openmetrics-text" http://<your application>/metrics | grep -i "traceid"
.
See also:
Related Enterprise Metrics resources
Running Prometheus-as-a-service with Grafana Enterprise Metrics
Introducing Grafana Enterprise Metrics (GEM), a simple and scalable Prometheus service that is seamless to use, simple to maintain, and supported by Grafana Labs.
How Robinhood scaled from 100M to 700M time series with Grafana Enterprise Metrics
In this GrafanaCONline session, the Robinhood team tells how GME (GameStop) led to GEM (Grafana Enterprise Metrics).
Benchmarking Grafana Enterprise Metrics for horizontally scaling Prometheus up to 500 million active series
We stress-tested GEM to show how it horizontally scaled. One takeaway: Hardware usage scales linearly up to 500 million active series.