Plugins 〉Grafana Enterprise Metrics

Application

grafana

Enterprise

Grafana Enterprise Metrics

Overview
Installation
Change log
Related content

Grafana Enterprise Metrics

Grafana Enterprise Metrics (GEM) is a commercial offering based on the open-source project Mimir. The commercial offering allows you to deploy a higly-scalable, simple, and reliable metrics cluster in your own data center. This app plugin gives you an easy way to manage your metrics cluster.

Features

Tenant management Easily scale up to hundreds of metrics tenants on a single Metrics Enterprise cluster.
Access policies and tokens Fully control and visualize who has access to what.
Ring health Give an overview on the current status of the various service rings in the cluster.

Dashboards

Self-monitoring GEM includes the ability to directly record self-monitoring metrics to allow you to easily monitor the health and stability of GEM itself. A set of self-monitoring dashboards is made available, using GEM's self-monitoring metrics to help monitor GEM system health.
Cardinality management GEM provides the ability to understand the cardinality of your metrics and labels using cardinality management dashboards to help visualize and explore the data from the cardinality management API.

Configuration

After the plugin has been installed into your Grafana instance, enable and initialize it:

Navigate to Administration > Plugins and data > Plugins
Select the Grafana Enterprise Metrics app from the list of plugins.
If an alert indicates that this app plugin is not enabled, enable the plugin by clicking on the Enable button. The alert text will indicate where it can be found.
Ensure that the Connection settings section is filled out using the bootstrap token for your GEM, and the corresponding URL where the cluster can be accessed by the Grafana instance.
Start using the Grafana Enterprise Metrics app plugin.

Installing Grafana Enterprise Metrics on Grafana Cloud:

For more information, visit the docs on plugin installation.

Changelog

All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning.

Shared versioning and change log

The content of this changelog as far back as October 2024 (v5.0.0) is shared by multiple plugins in the common "GEx Plugins" project. Changelog entries will call out whether a change is specific to an individual plugin. The releases for the plugins will be synchronized and share the same version numbers.

v5.1.2 -- December 10, 2024

[FIX] - Provide on-page navigation links to each of the main pages

v5.1.1 -- December 9, 2024

[FIX] - Crash on configuration page if the plugin was not already provisioned or configured

v5.1.0 -- December 7, 2024

[FEAT] - Added support for TLS certificates in connection configuration

v5.0.2 -- October 11, 2024

[FIX] - Corrected error message handling for unconfigured connection settings

v5.0.0 -- October 10, 2024

[BREAKING] - Support for Admin API v1 and v2 has been dropped. Only Admin API v3 will be used, which was introduced in April 2022.
- The earliest versions of the Enterprise Databases with the v3 API are as follows:
  - GEM v2.0
  - GEL v1.4
  - GET v1.3
[BREAKING] - The officially supported Grafana version range is now: >= 10.4.10 < 11 || >= 11.1.0
- Backward compatibility support for Grafana Enterprise v8 and v9 has been removed
[FIX] - Enterprise license failure detection and reporting has been improved to help identify this problem faster
[FIX] - Updated dependencies to resolve security concerns
[FIX] - Remove 5 second delay when saving plugin configuration

End of synchronized changelog

GEM plugin changes (pre-synchronization)

The following versions and changes represent those of the GEM plugin, prior to synchronizing all plugins to v5.0.0.

v4.1.2 -- June 25, 2024

[FIX] - Updated dependencies to resolve security concerns
[FIX] - Updated Cardinality dashboards for compatibility with Scenes-based dashboards

v4.1.1 -- April 10, 2024

[FIX] - Replace deprecated GEM self-monitoring dashboard panels to use current "timeseries" and "table" panel types

v4.1.0 -- March 14, 2024

[FIX] - Ring health: standardized look of ring selector, and removed hard-to-see alert coloring
[FIX] - Ring health: show "Alertmanager" in alphabetical order
[FIX] - Updated dependencies to resolve security concerns

v4.0.9 -- January 12, 2024

[FIX] - Detect more license error and backend error states to present root cause information to the user
[FIX] - Updated dependencies to resolve security concerns

v4.0.8 -- November 28, 2023

[FIX] - Updated dependencies to resolve security concerns

v4.0.7 -- November 07, 2023

[FIX] - Updated dependencies to resolve security concerns
[FIX] - Freeze GEM dashboards and remove dashboard generator scripts

v4.0.6 -- September 22, 2023

[FIX] - Updated dependencies to resolve security concerns

v4.0.5 -- September 19, 2023

[FIX] - Updated dependencies to resolve security concerns

v4.0.4 -- September 7, 2023

[ENHANCEMENT] - Tenant stats will refetch every 30 minutes automatically
[ENHANCEMENT] - Display details of Grafana Enterprise license on Licenses page
[FIX] - Addresses CVE vulnerability reports
[FIX] - Updated datasource proxies to use uid instead of id (cardinality)

v4.0.3 -- June 19, 2023

[FIX] - Handle tenants and access policies that do not have a display name

v4.0.1 -- March 31, 2023

[ENHANCEMENT] - Added selectable custom expiration date to token expiry
[ENHANCEMENT] - Formatted numbers according to local number string
[ENHANCEMENT] - Added indication on minimum access policy scope requirements for datasource creation after generating a token
[ENHANCEMENT] - The plugin overview page now displays accurate descriptions
[FIX] - When creating datasource, we check if backend URL is defined
[FIX] - When creating token in an access policy with no remaining tenants activated, the user is warned and not able to create datasources

v4.0.0 -- January 31, 2023

[FEATURE] - Changed supported Grafana version range: >= 8.5.20 < 9 || >= 9.1.0; warnings are displayed if the Grafana version is not within the officially supported range
[FEATURE] - Added support for Grafana 9.3, including Grafana's new navigation updates
[FEATURE] - The backend database version is now displayed on the plugin (and configuration page)
[FEATURE] - Added new self-monitoring dashboards:
- Overview
- Alertmanager (requires GEM v2.5 or later for all panels to function)
- Alertmanager resources
- Config
- Object Store
- Overrides
- Queries
- Ruler
- Tenants
- Top tenants
[ENHANCEMENT] - The cardinality dashboards have a more complete ad hoc filter feature set:
- The dashboard links retain existing context when drilling deeper through Cardinality Dashboards via ad hoc filters
- Additional links have been added (particularly to the tree map panels) that allow the user to drill down and explore cardinality in different scopes
[ENHANCEMENT] - The cardinality datasource has been enhanced:
- The ad hoc filters are respected for the various query type in the query editor and query explorer viewproperly scope "pairs"->"details" queries/panels, as well as "series"->"totals"
- Scoping can be done with the "selector" input
[ENHACENMENT] - Added descriptions for access policy scopes
[ENHACENMENT] - Added progress indicators during requests to create/edit tenants, access policies, and tokens; a successful alert notification now appears as on the top-right corner of the screen when the request is completed
[ENHANCEMENT] - The ring health page allows selection of rings that have no members
[ENHANCEMENT] - The plugin configuration page has been moved to exist alongside the other plugin pages (e.g., tenants, access policies, etc.); this manifests as a tab when "topnav" is disabled, or within the same navigation menu tree structure when "topnav" is enabled;
[ENHACNEMENT] - The plugin configuration page no longer duplicates the functionality of disabling or enabling the plugin
[ENHACNEMENT] - The plugin configuration page has a cancel button which reverts back to the saved state
[FIX] - The cardinality datasource has changed the dataframe heading for "labels"->"details" queries to show the label key
[FIX] - The cardinality datasource query type "series"->"subtotal" now correctly populates options for the "query target" field
[FIX] - The ring health page handles backend errors more gracefully (e.g., when the request from using the forget button fails)
[FIX] - The ring health page is able to handle the newer implementation for expressing "unhealthy" ringe member status
[FIX] - Access policies with very long names are better supported: it is now possible to create more than one token for them; their long names are now displayed without breaking the layout
[FIX] - The tenants page no longer makes multiple redundant requests (per number of tenants)
[FIX] - Admin resource display names are now converting all non-alphanumeric characters correctly into dashes when generating their identifier
[FIX] - Added restrictions to tenant limit overrides on input fields, such as preventing invalid negative limits

v3.6.4 -- January 31 2023

[FIX] - Replace automatic refresh with a refresh recommendation notification when dashboards and datasources are modified

v3.6.3 -- September 23 2022

[FIX] - Corrected some links on the "System monitoring overview" dashboard to the "Per-tenant usage"

v3.6.2 -- September 12 2022

[ENHANCEMENT] - Added the ability to use adhoc filters on the cardinality dashboards
[FIX] - Fixed tenant creation form saving behavior for the case where the user enters no overrides for limits

v3.6.1 -- August 10 2022

[FIX] - The self-monitoring data which enriches the more accurate tenant stats is correctly updating from the less accurate values

v3.6.0 -- July 28 2022

[FEATURE] - Access policies can set metrics:import scope for GEM clusters with TSDB block upload support (expected in GEM 2.3)
[ENHANCEMENT] - Use cluster feature detection for better backward and forward compatibility with backend
[FIX] - The editing of tenants and access policies now uses a edit button
[FIX] - The self-monitoring data which enriches the more accurate tenant stats has been corrected to match the results shown on the per-tenant dashboard
[FIX] - Self-monitoring compactor dashboard "last successful run" panels now render correctly in Grafana 8.5 and later
[FIX] - Self-monitoring overview dashboard "fraction" panels now render correctly in Grafana 9.0 and later
[FIX] - Ensure newly created access policies display correctly when running versions of GEM prior to 2.0 on versions of Grafana 8.4 and later
[FIX] - On access policies page, indicate token fetch status instead of always defaulting to a misleading "no tokens"
[FIX] - Fixed issues preventing the ability to edit access policy scopes
[FIX] - Removed query scheduler from ring health page since it is not present in GEM

v3.5.3 -- June 10 2022

[FIX] - Fixed dashboard update behavior which caused a misleading error message, blocking the update
[FIX] - Remove GEM dashboard folders when the GEM plugin is disabled (except for folders that still contain user-defined dashboards or are associated with alerting rules)
[FIX] - Various panels in the GEM self-monitoring dashboards had faulty selectors in their queries when running GEM instances with multiple targets (e.g., -target=all,overrides-exporter)
[FIX] - Clarified language on cluster configuration page, to better indicate the the origin of the displayed values
[FIX] - Cardinality dashboards: fix broken link find unused metrics to use mimirtool
[FIX] - The cluster configuration page now correctly extracts the top level configuration, and uses alphabetic order for all other sections

v3.5.2 -- May 2 2022

[FIX] - Fixed dashboard update behavior which was contributing to disabling the GEM plugin
[FIX] - When admin api version is not explicitly configured, the latest compatible version will be automatically detected and act as a default
[FIX] - The access policy creation form provides an informative tooltip for the "Yes" button (regarding scopes for creating a datasource); also ensuring the correct scopes are added by default
[FIX] - Fixed interpretation of dates in "last heartbeat" column in ring health page; the relative time offset will periodically update now, and a tooltip will display the reported date and time
[FIX] - Clarify license status, providing warning banners if the license is expired, or near expiry
[FIX] - Alphabetized ring types on ring health page
[FIX] - Fixed the ring health page "forget" button behavior: ring members will now only be removed from the list when the request has succeeded

v3.5.1 -- April 25 2022

[FIX] - Fixed errors causing the plugin to be disabled

v3.5.0 -- April 22 2022

[ENHANCEMENT] - Improved flow of access policy creation, including automatically selecting appropriate scopes based on user's intention to create a datasource
[ENHANCEMENT] - Added corrective measures to self-monitoring datasource when inconsistencies are detected, including protection against duplicate/similar datasource names, and manual user deletion/modification
[FIX] - Fixed token deletion for older versions of GEM
[FIX] - Fixed automatic dashboard datasource initialization so that it is limited to users with the Admin role
[FIX] - Prevent the plugin from being disabled after modifying the plugin configuration

v3.4.1 -- April 18 2022

[ENHANCEMENT] - Improved cardinality management dashboard layout for consistency and narrower window sizes
[FIX] - Fixed compatibility with admin API versions 1 and 2 for listing access policies and tokens
[FIX] - Fixed issue displaying tenant stats when self-monitoring datasource cannot be found
[FIX] - Fixed plugin configuration validator providing misleading error messages regarding available versions when a url error should have been shown

v3.4.0 -- April 13 2022

[FEATURE] - Added last successful run compactor panels in Self-monitoring compactor dashboards
[FEATURE] - Allow access policy tenant wildcards when creating tokens and datasources
[FEATURE] - Include panel for exemplars in Self-monitoring per tenant and Overview dashboards
[FEATURE] - Users can now set per-tenant compactor block retention and exemplar limits
[ENHANCEMENT] - Added Example label values to cardinality dashboard
[ENHANCEMENT] - Added Fraction of series that have the target label applied on the label cardinality dashboard
[ENHANCEMENT] - Added Percentage of series panel to cardinality labels dashboard
[ENHANCEMENT] - Added Sample label values to the metric cardinality dashboard
[ENHANCEMENT] - Added more rings to the Ring health page
[ENHANCEMENT] - Added self-monitoring data to display more accurate tenant stats
[ENHANCEMENT] - Added support for admin API versions 2 and 3; version can be selected in plugin configuration
[ENHANCEMENT] - Debug export now appears, conditionally on backend support
[ENHANCEMENT] - Improved form usability for creating and editing tenants, access policies, and tokens
[ENHANCEMENT] - Restored compatibility with Grafana v7.5
[ENHANCEMENT] - Token creation dialog now defines expiry in terms of days
[ENHANCEMENT] - When new version of GEM is installed, dashboards automatically refresh
[FIX] - Self-monitoring per-tenant usage dashboard better represents series limits
[FIX] - Fixed token creation dialog's copy-to-clipboard functionality for later versions of Grafana
[FIX] - Interpret initial If-Match ETag when fetching collections of tokens, access policies, and tenants
[FIX] - Lowered the refresh interval of the cardinality data
[FIX] - Remove extra trailing slashes in plugin settings and remote write URLs
[FIX] - Resolved bugs for cardinality datasource query editor controls
[FIX] - Self-monitoring datasources can now handle multiple Grafana instances accessing a common GEM instance
[FIX] - Token creation dialog will offer to create datasources only when read scopes are present
[FIX] - When GEM plugin is disabled, empty dashboard folders will not be created

v3.3.2 -- March 14 2022

[FIX] - Fixed copy-to-clipboard functionality for Grafana v8.3+

v3.3.1 -- February 24 2022

[FIX] - Add compatibility with grafana-enterprise v8.4.2+

v3.3.0 -- November 16 2021

[FEATURE] - Cardinality management dashboards
[FIX] - Change of terminology to "Tenants" (previously was "Instances")
[FIX] - Only customizable limits are shown when editing limits settings for tenants
[FIX] - Deleting a tenant's custom limits can be done be removing all custom limit entries before saving changes
[FIX] - Fixed issue related to saving and updating instance limits (If-Match, ETag stability)
[FIX] - Dashboard initialization routines refresh logic

v3.2.1 -- August 23 2021

[FIX] - Self-monitoring dashboards: sort a few charts in descending order

v3.2.0 -- August 19 2021

[ENHANCEMENT] - Show more detailed error messages for not correctly formatted access-tokens on the plugin config page
[ENHANCEMENT] - Update self-monitoring dashboards, introduce a new "Overview Dashboard"

v3.1.0 -- August 09 2021

[FEATURE] - Add a button for exporting debug information about the gateway / admin API target
[FEATURE] - Make it possible to create wildcard access-policies that are applied to all instances in a cluster
[FIX] - GEM Self-Monitoring datasource: handle redundant slash in backendUrl during initialization
[FIX] - GEM Self-Monitoring datasource: make existence detection work in proxied environments
[FIX] - Ring health table: use natural sort order when sorting by member ids (interpret numbers inside strings)
[FIX] - In Access Policy form, stop showing the __system__ instance in the instance-selector
[FIX] - When creating a token for an access policy which applies to multiple instances, there is now a button available to create a single data source to encompass all of them; Users can now create a datasource for doing cross-tenant queries in a single click

v3.0.4 -- June 28 2021

[FIX] - Prevent enter key from triggering label delete in "create / update access policy"

v3.0.3 -- June 17 2021

[FIX] - Updated scope names: rules:read and rules:write for compatibility with recent backend changes
[FIX] - Updated text content of GEM Self-Monitoring dashboards (introduced in v2.1.0)

v3.0.2 -- June 15 2021

[FIX] - Moved self-monitoring dashboards into separate folder (introduced in v2.1.0)
[FIX] - Improved flexibility of GEM Self-Monitoring dashboard initialization (introduced in v2.1.0)

v3.0.1 -- June 15 2021

[FIX] - Make the modals scrollable when creating a new instance, access-policy or token (introduced in v3.0.0)
[FIX] - Make updates work again by passing the correct If-Match headers (introduced in v3.0.0)

v3.0.0 -- June 14 2021

[FEATURE] - Make the plugin compatible with Grafana 8 (Breaking change! - this version doesn't work with earlier versions of Grafana anymore)

v2.1.1 - June 17 2021

[FIX] - Updated scope names: rules:read and rules:write for compatibility with recent backend changes
[FIX] - Updated text content of GEM Self-Monitoring dashboards (introduced in v2.1.0)

v2.1.0 - June 14 2021

[FEATURE] - Make access policies editable
[FEATURE] - Update the look of the access policies list items
[FEATURE] - Include GEM Self-Monitoring dashboards

v2.0.1 - April 29 2021

[FIX] - Make array-type instance limits editable
[FIX] - Make data source links respect the base URL after creating a data source for a token
[FIX] - Make "admin", "alerts:read" and "alerts:write" scopes available for new access policies
[FIX] - Remove the cluster selector for now under the "Ring health" page

v2.0.0 - April 20 2021

[FIX] - Make instance statistics show up consistently
[FEATURE] - Make instances editable
[FEATURE] - Visualise instance limits in the instances list
[FEATURE] - Add a way to set instance-specific limits when creating a new instance
[FEATURE] - A generic Ring Health page replaced the Ingester Ring page
- Now all five rings can be monitored and repaired
- The user can specify the refresh interval

v1.9.3 -- March 17 2021

[FIX] - Show hints (and disable the submit) for incorrectly formatted tokens on the app config page
[FIX] - Make backspaces work in the token input field

v1.9.2 -- February 18 2021

[FIX] - Update the README

v1.9.1 -- February 03 2021

[FIX] - Add the backend plugin to the build

v1.9.0 -- February 01 2021

[FEATURE] - Update the design for no instances / access policies
[FEATURE] - Display the current UTC time above the instances / access-policies lists
[FEATURE] - Add a simple way to automatically generate datasources when creating a new metrics:read token

v1.8.0 -- January 22 2021

[FEATURE] - Make it possible to set the expiration date for a new token
[FEATURE] - Update the lookout of the successful token creation modal
[FIX] - Only show the label selectors when they make sense while creating an access policy

v1.7.0 -- January 22 2021

[FEATURE] - Introduce all scopes supported by the backend when creating a new access policy
[FIX] - Fix flakiness of the auto-generated name when creating a new access policy
[FIX] - Don't touch hyphens "-" when auto-generating the name for instances and access policies

v1.6.0 -- January 20 2021

[FEATURE] - Show copyable remote-write config snippets when creating a new token for an access policy with write permissions

v1.5.0 -- January 12 2021

[FEATURE] - Make it possible to set the "name" manually for a new instance

v1.4.2 -- January 07 2021

[FIX] - Fix the sidebar navigation

v1.4.1 -- December 16 2020

[FIX] - Don't show the placeholder for label selectors if they are not defined for an instance
[FIX] - Fix the width of the instance names on the detailed access policy page

v1.4.0 -- December 16 2020

[FEATURE] - Make it possible to define label selectors for the instances when creating an access policy
[FEATURE] - Add a detailed page for access policies with syntax highlighted label policies

v1.3.0 -- November 19 2020

[FEATURE] - Show collapsable sections under the cortex config and make it searchable
[FEATURE] - Mark the default access policy and remove the delete button from it. Also add a hint on how to delete it.

v1.2.0 -- November 2 2020

[FEATURE] - Add a tab for showing license information
[FEATURE] - Add a new design for access policies
[FEATURE] - Show instance statistics if they are available

v1.1.1 -- September 24 2020

[FIX] - Make plugin navigation links work even if Grafana is installed under a base URL
[FIX] - Make error messages coming from the API visible for the user

v1.1.0 -- September 16 2020

[FEATURE] - Update the logo

v1.0.0 -- September 16 2020

[FEATURE] - Introduce access policies
[FEATURE] - Switch out Keys with access policies & tokens
[FEATURE] - Restrict the plugin and the admin API to only be accessible by Admin users
[FEATURE] - Remove the datasource plugin and move all configuration under the app plugin
[FEATURE] - Move the tokens under the access policies tab for an easier user flow
[CHORE] - Add an optional mock server for easier development

v0.9.1 -- August 13 2020

[FIX] Enable the Ingester Ring tab by default

v0.9.0 -- August 13 2020

[FEATURE] Add an Ingester Rings tab showing a basic table with the available ingesters
[FEATURE] Update the plugin logo
[CHORE] Add plugin description to the plugin config page