Enable multi-cluster support (optional)
The Kube-Prometheus Helm chart offers support for some multi-cluster use cases. You may need to adjust some dashboard panels, recording rules, and alerting rules for your specific architecture. These adjustments go beyond the scope of this guide.
Add a cluster
label to rule aggregations
If you want to modify recording and alerting rules to aggregate metrics over a cluster
label, you can use cortex-tools’ rules prepare
command to add a cluster
label to a set of recording and alerting rules. You can modify the aggregation label using the -l
flag. The default label is cluster
. If you use this command, ensure that you’ve set the corresponding aggregation label using external_labels
in your Prometheus configuration. (If you’ve completed the Migrate a Kube-Prometheus Helm stack guide, you’ve already set this to cluster: test
).
Extract the rules from the Prometheus container as demonstrated in the Import recording and alerting rules guide.
Run the following command in the directory containing the rules files:
mimirtool rules prepare -i *.yaml
The
-i
flag modifies the rules files in place. You can omit this flag to test the command. This command modifies many rules to keep or add acluster
label to aggregations, which enables multi-cluster support.
You will then need to load the rules into Grafana Cloud Alerting as demonstrated in the Import recording and alerting rules guide.
Enable dashboard multi-cluster support
To enable dashboard multi-cluster support:
Modify your
values.yaml
Helm chart configuration file to enable multi-cluster support for Grafana dashboards:. . . grafana: sidecar: dashboards: multicluster: true
Roll out the changes using
helm upgrade
:helm upgrade -f values.yaml your_release_name prometheus-community/kube-prometheus-stack
To learn more, see the Kube Prometheus Helm chart’s values.yaml
file.
You will need to repeat the import dashboards step from the Migrate a Kube-Prometheus Helm stack to Grafana Cloud guide to import these modified dashboards to Grafana Cloud.