Menu

Important: This documentation is about an older version. It's relevant only to the release noted, many of the features and functions have been updated or replaced. Please view the current version.

DocumentationGrafana Enterprise Metrics DocumentationAdmin API
Enterprise

Admin API documentation

NOTE: Admin API version v1 endpoints /admin/api/v1/ are deprecated and will be removed in a future release of GEM. Use the new Admin API version v2 endpoints /admin/api/v2/ instead.

In addition to the standard Cortex API endpoints, Grafana Enterprise Metrics supports an admin HTTP API for managing cluster resources such as tenants and tokens.

API operations

The API supports standard CRUD (create, read, update, and delete) operations for most resources. Creates are issued via POST requests, reads via GET requests, updates via PUT requests, and deletes via DELETE requests. When running multiple instances of the Admin API, we recommend that you enable leader election so that all mutations occur on a single, leader-elected instance.

Write consistency (If-Match and ETag headers)

To ensure concurrent changes are propagated in a way that avoids unintentional overwrites, an If-Match header is required for all requests that mutate data. The current version of a resource to be sent in the If-Match header can be found in the ETag header returned on all GET and PUT requests that return a single resource in the response body.

Here are a few examples of how to get the value of the ETag header using curl. The first example uses the argument -D - to write all headers to standard out, and then parses for the ETag header.

console 
curl -u :$API_TOKEN -sD - -o /dev/null http://localhost:8080/admin/api/v2/accesspolicies/{name} | grep -i ETag | cut -d " " -f 2

This second example uses the -i option to print the headers along with the response payload.

console 
curl -u :$API_TOKEN -i http://localhost:8080/admin/api/v2/accesspolicies/{name}

When making a request to mutate data, make sure to include the double quotes (") around the value in the If-Match header. So, for example, the following is a request to delete the access policy named the-access-policy, currently at version 1:

console 
curl -u :$API_TOKEN -X DELETE -H 'If-Match: "1"' http://localhost:8080/admin/api/v2/accesspolicies/the-access-policy

Optionally, a wildcard "*" may be passed as the If-Match version which effectively disables the write consistency protections and allows any current version to be updated.

Cluster API

List clusters

GET /admin/api/v1/clusters

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/clusters instead.

GET /admin/api/v2/clusters

Response:

json 
{
  "items": [
    {
      "name":  "<string>",              // cluster name
      "display_name": "<string>",       // cluster display name
      "created_at": "<rfc3339 string>", // date created 
      "kind": "<string>",               // cluster type, currently only cortex is supported
      "base_url": "<string>"            // Base URL of the cluster, optional
    }
    ...
  ],
  "type": "cluster"
}

Example:

console 
$ curl -u :$API_TOKEN http://localhost:8080/admin/api/v2/clusters | jq
{
  "items": [
    {
      "name": "metrics-enterprise-dev",
      "display_name": "metrics-enterprise-dev",
      "created_at": "2020-07-13T16:50:41.953793Z",
      "kind": "cortex",
      "base_url": ""
    }
  ],
  "type": "cluster"
}

Get cluster

GET /admin/api/v1/clusters/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/clusters/{name} instead.

GET /admin/api/v2/clusters/{name}

NOTES:

  • There is no ETag header on a cluster response because clusters are immutable.

Response:

json 
{
  "name":  "<string>",              // cluster name
  "display_name": "<string>",       // cluster display name
  "created_at": "<rfc3339 string>", // date created 
  "kind": "<string>",               // cluster type, currently only cortex is supported
  "base_url": "<string>"            // Base URL of the cluster, optional
}

Example:

console 
$ curl -u :$API_TOKEN http://localhost:8080/admin/api/v2/clusters/metrics-enterprise-dev | jq
{
  "name": "metrics-enterprise-dev",
  "display_name": "metrics-enterprise-dev",
  "created_at": "2020-07-13T16:50:41.953793Z",
  "kind": "cortex",
  "base_url": ""
}

Tenant API

Model

The tenant model is made up of the following fields:

  • name: The machine-readable name of a tenant. It must be between 3 and 64 characters and only include the following characters, [a-z0-9-_]. Once set, this field is immutable.
  • display_name: The human-readable name of a tenant. It can contain any set of characters and can be changed.
  • status: The current status of this tenant. It can have the following values:
    • active
    • inactive
    • unknown
  • cluster: The name of the Grafana Enterprise Metrics cluster that this tenant is scoped to. You will only be able to write/query this tenant from this cluster.
  • limits: Tenant specific resource usage limits. Limits are key-value pairs (JSON object) that correspond to Cortex limits configuration. This field is optional and will be omitted from responses when unset.

List tenants

GET /admin/api/v1/instances

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/tenants instead.

GET /admin/api/v2/tenants

Response:

json 
{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "status": "<string>",
      "cluster": "<string>",
      "limits": {
        ...
      }
    },
    ...
  ],
  "type": "tenant"
}

Example:

console 
$ curl -u :$API_TOKEN localhost:8080/admin/api/v2/tenants | jq
{
  "items": [
    {
      "name": "metrics-enterprise-dev",
      "display_name": "Grafana Enterprise Metrics Dev Tenant",
      "created_at": "2020-07-13T17:37:59.341728283Z",
      "status": "active",
      "cluster": "metrics-enterprise-dev",
      "limits": {
        "ingestion_rate": 350000,
        "ingestion_burst_size": 350000,
        "max_series_per_query": 100000,
        "max_global_series_per_user": 300000,
        "max_global_series_per_metric": 300000,
        "ruler_max_rules_per_rule_group": 0,
        "ruler_max_rule_groups_per_tenant": 0
      }
    }
  ],
  "type": "tenant"
}

Create tenant

POST /admin/api/v1/instances

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v2/tenants instead.

POST /admin/api/v2/tenants

Example:

console 
curl -u :$API_TOKEN localhost:8080/admin/api/v2/tenants \
--data '{"name":"metrics-enterprise-dev", "display_name":"Grafana Enterprise Metrics Dev Tenant", "cluster": "metrics-enterprise-dev"}'

Update tenant

PUT /admin/api/v1/instances/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint PUT /admin/api/v2/tenants/{name} instead.

PUT /admin/api/v2/tenants/{name}

NOTES:

  • The name field of the object in the request body is ignored during updates.
  • The created_at field may not be modified during updates.
  • If-Match header matching the current version is required.

Response:

json 
{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "status": "<string>",
  "cluster": "<string>",
  "limits": {
    ...
  }
}

Example:

console 
curl -u :$API_TOKEN localhost:8080/admin/api/v2/tenants/metrics-enterprise-dev \
-X PUT -H 'If-Match: "123"` \
--data '{"display_name":"Grafana Enterprise Metrics Dev Tenant", "cluster": "metrics-enterprise-dev"}' | jq
{
  "name": "metrics-enterprise-dev",
  "display_name": "Grafana Enterprise Metrics Dev Tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "active",
  "cluster": "metrics-enterprise-dev",
  "limits": {
    "ingestion_rate": 350000,
    "ingestion_burst_size": 350000,
    "max_series_per_query": 100000,
    "max_global_series_per_user": 300000,
    "max_global_series_per_metric": 300000,
    "ruler_max_rules_per_rule_group": 0,
    "ruler_max_rule_groups_per_tenant": 0
  }
}

Get tenant

GET /admin/api/v1/instances/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/tenants/{name} instead.

GET /admin/api/v2/tenants/{name}

NOTES:

  • The current version of the tenant will be returned in an ETag header on the response.

Response:

json 
{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "status": "<string>",
  "cluster": "<string>",
  "limits": {
    ...
  }
}

Example:

console 
$ curl -u :$API_TOKEN localhost:8080/admin/api/v2/tenants/metrics-enterprise-dev | jq
{
  "name": "metrics-enterprise-dev",
  "display_name": "Grafana Enterprise Metrics Dev Tenant",
  "created_at": "2020-07-13T17:37:59.341728283Z",
  "status": "active",
  "cluster": "metrics-enterprise-dev",
  "limits": {
    "ingestion_rate": 350000,
    "ingestion_burst_size": 350000,
    "max_series_per_query": 100000,
    "max_global_series_per_user": 300000,
    "max_global_series_per_metric": 300000,
    "ruler_max_rules_per_rule_group": 0,
    "ruler_max_rule_groups_per_tenant": 0
  }
}

Delete tenant

DELETE /admin/api/v1/instances/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint DELETE /admin/api/v2/tenants/{name} instead.

DELETE /admin/api/v2/tenants/{name}

NOTES:

  • If-Match header matching the current version is required.

Example:

console 
curl -X "DELETE" -u :$API_TOKEN -H 'If-Match: "123"' localhost:8080/admin/api/v2/tenants/metrics-enterprise-dev

Access policy API

List access policies

GET /admin/api/v1/accesspolicies

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/accesspolicies instead.

GET /admin/api/v2/accesspolicies

Response:

json 
{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "realms": [
        {
          "tenant": "<string>",
          "cluster": "<string>"
        }
      ],
      "scopes": [
        "<string>",
        ...
      ]
    },
    ...
  ],
  "type": "access_policy"
}

Create access policy

POST /admin/api/v1/accesspolicies

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v2/accesspolicies instead.

POST /admin/api/v2/accesspolicies

Payload:

json 
{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Realms

Realms designate the tenant/cluster pairs that the access policy allows requests for:

  • tenant: This field must be set to an existing tenant or *. * denotes access to all tenants.
  • cluster: This field must be set to an existing cluster.
  • label_policies: This is an optional field that can be set to a list of label policies. An access policy gets access to metrics that match at least one of the label policies. If label policies are not set, all metrics will match.
Label policy, selector, and label matchers

A label policy contains a selector, that consists of a list of label matchers.

The following label matcher types are supported:

  • EQ: Select labels that are exactly equal to the provided string.
  • NEQ: Select labels that are not equal to the provided string.
  • RE: Select labels that regex-match the provided string.
  • NRE: Select labels that do not regex-match the provided string.
json 
{
  "selector": [
    {
      "type": "<enum: EQ | NEQ | RE | NRE>",
      "name": "<string>",
      "value": "<string>"
    }
  ]
}

The PromQL selector {job="grafana-labs", role!="software-engineer"}, would translate into this label policy:

json 
{
  "selector": [
    {
      "type": "EQ",
      "name": "job",
      "value": "grafana-labs"
    },
    {
      "type": "NE",
      "name": "role",
      "value": "software-engineer"
    }
  ]
}

Scopes

Scopes designate what operations tokens assigned to this access policy will be able to do when calling the GEM API.

  • metrics:read: Permission to view data from a tenant
  • metrics:write: Permission to write data to a tenant
  • metrics:delete: Permission to delete data from a tenant
  • rules:read: Permission to read ruler rules
  • rules:write: Permission to write ruler rules
  • admin: Permission to perform admin operations

Update access policy

PUT /admin/api/v1/accesspolicies/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint PUT /admin/api/v2/accesspolicies/{name} instead.

PUT /admin/api/v2/accesspolicies/{name}

NOTES:

  • If-Match header matching the current version is required.
  • Update differs from create in that only the Display Name, Realms, and Scopes are updatable.

Payload:

json 
{
  "display_name": "<string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Get access policy

GET /admin/api/v1/accesspolicies/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/accesspolicies/{name} instead.

GET /admin/api/v2/accesspolicies/{name}

NOTES:

  • The current version of the access policy will be returned in an ETag header on the response.

Response:

json 
{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "realms": [
    {
      "tenant": "<string>",
      "cluster": "<string>"
    }
  ],
  "scopes": [
    "<string>",
    ...
  ]
}

Delete access policy

DELETE /admin/api/v1/accesspolicies/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint DELETE /admin/api/v2/accesspolicies/{name} instead.

DELETE /admin/api/v2/accesspolicies/{name}

NOTES:

  • If-Match header matching the current version is required.

Example:

console 
$ curl -X "DELETE" -u :$API_TOKEN -H 'If-Match: "123"' localhost:8080/admin/api/v2/accesspolicies/dev-read-write

Token API

List tokens

GET /admin/api/v1/tokens

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/tokens instead.

GET /admin/api/v2/tokens

Response:

json 
{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "expiration": "<rfc3339 string>",
      "access_policy": "<string>"
    },
    ...
  ],
  "type": "key"
}

Create token

POST /admin/api/v1/tokens

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint POST /admin/api/v2/tokens instead.

POST /admin/api/v2/tokens

Payload:

json 
{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "expiration": "<rfc3339 string>",
  "access_policy": "<string>"
}

Get token

GET /admin/api/v1/tokens/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/tokens/{name} instead.

GET /admin/api/v2/tokens/{name}

NOTES:

  • The current version of the token will be returned in an ETag header on the response.

Response:

json 
{
  "name": "<string>",
  "display_name": "<string>",
  "created_at": "<rfc3339 string>",
  "expiration": "<rfc3339 string>",
  "access_policy": "<string>"
}

Delete token

DELETE /admin/api/v1/tokens/{name}

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint DELETE /admin/api/v2/tokens/{name} instead.

DELETE /admin/api/v2/tokens/{name}

NOTES:

  • If-Match header matching the current version is required.

Example:

console 
curl -X "DELETE" -u :$API_TOKEN -H 'If-Match: "123"' localhost:8080/admin/api/v2/token/dev-read-write

License API

List licenses

GET /admin/api/v1/licenses

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/licenses instead.

GET /admin/api/v2/licenses

Response:

json 
{
  "items": [
    {
      "name": "<string>",
      "display_name": "<string>",
      "created_at": "<rfc3339 string>",
      "token": {
        "jti": "<string>",
        "iss": "<string>",
        "sub": "<string>",
        "iat": "<int64>",
        "exp": "<int64>",
        "nbf": "<int64>",
        "lexp": "<int64>",
        "lid": "<string>",
        "max_users": "<int64>",
        "included_admins": "<int64>",
        "included_viewers": "<int64>",
        "lic_exp_warn_days": "<int64>",
        "prod": [
          "<string>",
          ...
        ],
        "company": "<string>",
        "slug": "<string>",
      }
    },
    ...
  ],
  "type": "license"
}

Feature detection API

Get product and feature information

GET /admin/api/v1/features

Deprecated: this endpoint is deprecated and will be removed in a future release of GEM. Use the endpoint GET /admin/api/v2/features instead.

GET /admin/api/v2/features

Response:

json 
{
  "product": "<string>",
  "product_version": "<string>",
  "features": {
      "<string>": "<string>",
      ...
  }
}

Example response:

json 
{
  "product": "GEM",
  "product_version": "1.1",
  "features": {
      "lbac": "v2",
      "self_monitoring": "v1"
  }
}