Metrics Enterprise Config

Config

Grafana Metrics Enterprise utilizes a configuration file that is a superset of the Cortex configuration file.

This page gives an overview over the extensions that are available in the Grafana Metrics Enterprise configuration:

Note: The configuration of the getting started guide can be used as a starting point for writing your customised cluster configuration

Admin backend storage

Grafana Metrics Enterprise targets object storage as a dependency for using the Admin API for instance management and token authentication. This means a supported object store backend is required. The client that is used is configured using the same configuration options as configuring a blocks storage backend. All of the clients supported for blocks storage are also supported for the admin API.

Below are snippets that can be included in your configuration to setup each of the common object storage backends:

S3 / Minio Backend

admin_client:
  storage:
    type: s3
    s3:
      # The S3 bucket endpoint. It could be an AWS S3 endpoint listed at
      # https://docs.aws.amazon.com/general/latest/gr/s3.html or the address of an
      # S3-compatible service in hostname:port format.
      [endpoint: <string> | default = ""]

      # S3 bucket name
      [bucket_name: <string> | default = ""]

      # S3 secret access key
      [secret_access_key: <string> | default = ""]

      # S3 access key ID
      [access_key_id: <string> | default = ""]

      # If enabled, use http:// for the S3 endpoint instead of https://. This could
      # be useful in local dev/test environments while using an S3-compatible
      # backend storage, like Minio.
      [insecure: <boolean> | default = false]

Dynamic Headers

To configure dynamic HTTP headers for the admin storage’s S3 / Minio backend, the following configuration can be used. Note that the poll interval controls how often the file is reloaded and there is a section on the format of the dynamic headers file.

admin_client:
  type: s3
  storage:
    s3:
      header_map_file_path: <path to header file>
      header_map_poll_interval: <duration string>

GCS Backend

admin_client:
  storage:
    type: s3
    gcs:
      # GCS bucket name
      [bucket_name: <string> | default = ""]

      # JSON representing either a Google Developers Console client_credentials.json
      # file or a Google Developers service account key file. If empty, fallback to
      # Google default logic.
      [service_account: <string> | default = ""]

Blocks Storage

Grafana Metrics Enterprise extends Cortex’s blocks storage configuration by those features:

Rate limiting of List calls

List calls to a storage bucket can be rate limited. The blocks-storage.bucket-rate-limit.limit configuration option specifies how often per second a list call can be made before being rate limited. The blocks-storage.bucket-rate-limit.burst configuration option specifies the burst size. If blocks-storage.bucket-rate-limit.limit is smaller than or is equal to zero (the default) no rate limiting is applied. The snippet below shows how this can be set in your configuration file for an S3 backend:

blocks_storage:
  backend:                 s3
  bucket_rate_limit:       100
  bucket_rate_limit_burst: 1

Dynamic Headers for S3 / Minio Backend

To configure dynamic HTTP headers for the blocks storage’s S3 / Minio backend, the following configuration can be used. Note that the poll interval controls how often the file is reloaded and there is a section on the format of the dynamic headers file.

blocks_storage:
  backend:                 s3
  s3:
    header_map_file_path: <path to header file>
    header_map_poll_interval: <duration string>

Ruler

The Ruler is an optional service executing PromQL queries for recording rules and alerts. The ruler requires a backend storing the recording rules and alerts for each tenant.

Grafana Metrics Enterprise extends Cortex’s upstream ruler configuration by those features:

Remote-write forwarding

The following configuration options can be used to enable remote write rule groups and specify a desired directory to store the generated write-ahead log (WAL). To learn more see the remote-write rule forwarding documentation

enterprise_features:
  ruler_remote_write:
    enabled: <bool. defaults to false>
    wal_dir: <directory path. defaults to ./wal>

Dynamic Headers for S3 / Minio Backend

To configure dynamic HTTP headers for the ruler’s S3 / Minio backend, the following configuration can be used. Note that the poll interval controls how often the file is reloaded and there is section on the format of the dynamic headers file.

enterprise_features:
  ruler_s3_request_headers:
    file: <filename>
    poll_interval: <duration. defaults to 1m>

Dynamic Headers Configuration

A dynamic headers configuration file allows to configure HTTP headers, which can be dynamically changed during the runtime of Grafana Metrics Enterprise. It uses the YAML syntax and its format looks like this:

<header-name>: "header-value"
<second-header-name>: "second-header-value"

An example of specifying a bearer token for authentication could look like this:

Authorization: "Bearer my-secret-bearer-token"