---
title: "Use tenant federation | Grafana Enterprise Metrics documentation"
description: "Use tenant federation GEM supports creating access policies that can span multiple tenants. Doing so enables viewers in Grafana Enterprise to view data coming from more than one tenant simultaneously."
---

[Documentation](/docs/)![breadcrumb arrow](/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg) [Grafana Enterprise Metrics](/docs/enterprise-metrics/v2.17.x/)![breadcrumb arrow](/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg) [Manage](/docs/enterprise-metrics/v2.17.x/manage/)![breadcrumb arrow](/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg) [Manage a tenant](/docs/enterprise-metrics/v2.17.x/manage/tenant-management/)![breadcrumb arrow](/static/assets/img/icons/grafana-icon-breadcrumb-arrow-gray.svg) Use tenant federation

Enterprise

## Use tenant federation

GEM supports creating access policies that can span multiple tenants. Doing so enables viewers in Grafana Enterprise to view data coming from more than one tenant simultaneously.

This page will cover the ability to query data from multiple tenants at once, as well as the ability to create alerting and recording rules that use data from multiple tenants at once.

## Prerequisites

- A configured Grafana Enterprise Metrics cluster. To create a GEM cluster, refer to [Set up GEM](/docs/enterprise-metrics/latest/set-up/).
- This guide will assume there are two tenants: `team-engineering` and `team-finance`. To create a tenant, refer to [Set up a GEM tenant](/docs/enterprise-metrics/latest/set-up/set-up-gem-tenant/).

## Cross-tenant queries

### Set up an access policy with tenant federation and a token

To allow queries to span both GEM tenants, which are for demonstration purposes named `team-engineering` and `team-finance`, create a new access policy called `leadership`. The necessary steps are:

1. Create a new access policy `leadership`.
2. Enable the `metrics:read` scope.
3. Add the tenants `team-engineering` and `team-finance`. Alternatively, you can add the special tenant name `*` to create an access policy that has access to all tenants in the cluster.
4. Create a new token for the access-policy and store the token in your clipboard:

There’s supposed to be a video here, but for some reason there isn’t. Either we entered the id wrong (oops!), or Vimeo is down. If it’s the latter, we’d expect they’ll be back up and running soon. In the meantime, [check out our blog](/blog/)!

### Set up a Grafana data source using the access policy

1. Create a new Prometheus data source from the Grafana configuration menu.
2. Enter the URL of your GEM cluster, for example `http://enterprise-metrics/prometheus`.
3. From the **Auth** section, enable Basic auth.
4. In the **User** field, enter: `team-engineering|team-finance` where all the names of the tenants that you want to query across are separated by the `|` pipe character.
5. In the **Password** field, paste the token created in the token creation process.

Queries that are performed using this data source in either Explore or inside of dashboards are performed across all of the tenants that you specified in the **User** field, and are processed as if all of the data were in a single tenant.

To submit a query across all tenants that your access policy has access to, you can either:

1. Explicitly set the name of all the tenants separated by a pipe character “|” in the *username*. For example, to query across `tenant1`, `tenant2`, and `tenant3` you would enter `tenant1|tenant2|tenant3`.
2. Set the username to a wildcard character `*`. This will query all tenants that the access policy grants you access to, without requiring you to explicitly specify their names.

When using an access policy that has a wildcard (`*`) as the *username*, you can query all tenants for that cluster by also specifying `*` as the *username* in your data source URL.

Conversely, if you use a wildcard *username* in your data source configuration with an access policy with specific tenants, that data source has access to only those tenants.

There’s supposed to be a video here, but for some reason there isn’t. Either we entered the id wrong (oops!), or Vimeo is down. If it’s the latter, we’d expect they’ll be back up and running soon. In the meantime, [check out our blog](/blog/)!

## Cross-tenant alerting and recording rules

GEM supports alerting and recording rules that operate on data from multiple tenants.

To create a federated alerting or recording rule, use the `source_tenants` field when defining your rule group. This field defines which tenants the rule will read data from. Any rule in a group with a non-empty `source_tenants` field is considered a federated rule.

Using the same example tenants we defined above, we’ll create a federated rule group that evaluates data coming from tenants `team-engineering` and `team-finance`, and stores the resulting series in `team-engineering`.

First, enable query tenant federation and rules tenant federation. The configuration parameters for each are as follows:

Expand table

| CLI argument                            | YAML parameter                          |
|-----------------------------------------|-----------------------------------------|
| `-ruler.tenant-federation.enabled=true` | `ruler.tenant_federation.enabled: true` |
| `-tenant-federation.enabled=true`       | `tenant_federation.enabled: true`       |

If you are using the `mimir-distributed` Helm chart to deploy GEM, then you can enable these parameters by including the following configuration in your values file:

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
mimir:
  structuredConfig:
    tenant_federation:
      enabled: true
    ruler:
      tenant_federation:
        enabled: true
```

Next, create your rule and rule group in a file named `groups.yaml`.

YAML ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```yaml
namespace: fed-rule-group
groups:
  - name: example
    interval: 5m
    source_tenants: [team-engineering, team-finance]
    rules:
      - record: job:http_inprogress_requests:sum
        expr: sum by (job) (http_inprogress_requests)
```

This example recording rule is calculating a sum of `http_inprogress_requests` by job every 5 minutes. Because the source tenants are `team-engineering` and `team-finance`, GEM will use all series with metric name `http_inprogress_requests` in both of these tenants when calculating the sum.

To create the cross-tenant rule group, use the following [mimirtool](/docs/mimir/latest/operators-guide/tools/mimirtool/#rules) command:

sh ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy

```sh
mimirtool rules load --address http://enterprise-metrics/ --user team-engineering --key $API_TOKEN groups.yaml
```

The username used in the mimirtool command call is what dictates where the output of the rule will be stored. In our example above, the output of the rule will be stored in `team-engineering` because we have `--user team-engineering`.

The access policy to which the API token belongs must have `metrics:read` and `rules:write` scopes and apply to all the tenants in `source_tenants` (`team-engineering` and `team-finance` in our example) and the owning tenant (`team-engineering` in our example). Otherwise, an HTTP unauthorized status (`401`) is returned and the rule group is not created.

> **Note:** Creating federated rules is not supported with OIDC-based tokens that reference more than one GEM access policy. To create federated rules, either use a GEM token or an OIDC-based token with only a single GEM access policy that has `metrics:read` and `rules:write` scopes and applies to the owning tenant and the source tenants.

When the access policy used to create the federated rule group is deleted or updated to remove the `metrics:read` scope or to remove any of the `source_tenants`, that rule group will stop evaluating at the next rules synchronization. The synchronization interval is set by the `-ruler.poll-interval` configuration parameter. By default, this is 1 minute.

You can find more details about cross-tenant rules in the [Grafana Mimir API documentation](/docs/enterprise-metrics/latest/api/mimir-api/#set-rule-group).