Documentation Plugins Amazon CloudWatch Query editor

Amazon CloudWatch query editor

Grafana provides a query editor for the CloudWatch data source, which allows you to query, visualize, and alert on logs and metrics stored in Amazon CloudWatch. It is located on the Explore page. For general documentation on querying data sources in Grafana, refer to Query and transform data.

Choose a query editing mode

The CloudWatch data source can query data from both CloudWatch metrics and CloudWatch Logs APIs, each with its own specialized query editor.

• CloudWatch Metrics
• CloudWatch Logs

Select the API to query using the drop-down to the right of the Region setting.

CloudWatch Metrics query editor components

The following are the components of the CloudWatch query editor.

Use Builder mode

Create a query in Builder mode:

1. Browse and select a metric namespace, metric name, filter, group, and order options using information from the Metrics Insights keywords table.
2. For each of these keywords, choose from the list of available options.

Grafana constructs a SQL query based on your selections.

Use Code mode

You can also write your SQL query directly in a code editor by using Code mode.

The code editor includes a built-in autocomplete feature that suggests keywords, aggregations, namespaces, metrics, labels, and label values. Suggestions appear after typing a space, comma, or dollar ($) character, or by pressing CTRL+Space.

To run the query, click Run query above the code editor.

Metrics editor

When you select Builder mode within the Metric search editor, a new Account field is displayed. Use the Account field to specify which of the linked accounts to target for the given query. By default, the All option is specified, which will target all linked accounts.

While in Code mode, you can specify any math expression. If the Monitoring account badge displays in the query editor header, all SEARCH expressions entered in this field will be cross-account by default. You can limit the search to one or a set of accounts, as documented in the AWS documentation.

Logs editor

The Log group selector specifies the target log groups for the logs query. When the Monitoring account badge appears in the query editor header, you can search and select log groups across multiple accounts. Use the Account field to filter log groups by account. For large numbers of log groups, use prefix search to narrow the selection.

Query CloudWatch metrics

You can create two types of queries in the CloudWatch query editor:

• Metric Search, which help you retrieve and filter available metrics.
• Metric Query, which use the Metrics Insights feature to fetch time series data.

The query type you use depends on how you want to interact with AWS metrics. Use the drop-down in the upper middle of the query editor to select which type you want to create.

Metric Search queries

Metric search queries help you discover and filter available metrics. These queries use wildcards and filters to find metrics without needing to know exact metric names.

A valid metric query requires a specified namespace, metric name, and at least one statistic. Dimensions are optional, but if included, you must provide both a key and a value.

The Match Exact option controls how dimension filtering is applied to metric queries. When you enable match exact, the query returns only metrics whose dimensions precisely match the specified criteria.

This requires the following:

• All dimensions present on the target metric must be explicitly specified.
• Dimensions you don’t want to filter by must use a wildcard (*) filter.
• The metric schema must match exactly as defined in the CloudWatch metric schema documentation.

When Match Exact is disabled, you can specify any subset of dimensions for filtering. The query returns metrics that:

• Match the specified namespace and metric name.
• Match all defined dimension filters.
• May contain additional dimensions beyond those specified.

This mode provides more flexible querying but may return metrics with unexpected additional dimensions.

The data source returns up to 100 metrics matching your filter criteria.

Enhance metric queries using template variables to create dynamic, reusable dashboards.

Create dynamic queries with dimension wildcards

Use the asterisk (*) wildcard for dimension values to create dynamic queries that automatically monitor changing sets of AWS resources.

The query returns the average CPU utilization for all EC2 instances in the default region. With Match Exact disabled and InstanceId using a wildcard, the query retrieves metrics for any EC2 instance regardless of additional dimensions.

Auto-scaling events add new instances to the graph without manual instance ID tracking. This feature supports up to 100 metrics.

Click the Query inspector button and select Meta Data to see the search expression that’s automatically built to support wildcards.

To learn more about search expressions, refer to the CloudWatch documentation. The search expression is defined by default in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example, the query returns only metrics with exactly one dimension containing the name InstanceId.

Disabling Match Exact includes metrics with additional dimensions and creates a search expression even without wildcards. Grafana searches for any metric matching at least the namespace, metric name, and all defined dimensions.

Use multi-value template variables

When defining dimension values based on multi-valued template variables, the data source uses a search expression to query for the matching metrics. This enables the use of multiple template variables in one query, and also lets you use template variables for queries that have the Match Exact option disabled.

Search expressions are limited to 1,024 characters, so your query might fail if you have a long list of values. We recommend using the asterisk (*) wildcard instead of the All option to query all metrics that have any value for a certain dimension name.

Multi-valued template variables are supported only for dimension values. Using multi-valued template variables for Region, Namespace, or Metric Name is not supported.

Use metric math expressions

Create new time series metrics using mathematical functions on CloudWatch metrics. This supports arithmetic operators, unary subtraction, and other functions. For available functions, refer to AWS Metric Math.

To apply arithmetic operations, assign a unique string ID to the raw metric, then reference this ID in the Expression field of the new metric.

Period macro

If you use a CloudWatch SEARCH expression, consider using the $__period_auto macro rather than specifying a period explicitly. The $__period_auto macro will resolve to a CloudWatch period that is suitable for the chosen time range.

Create queries for alerting

Alerting requires queries that return numeric data, which CloudWatch Logs supports. For example, you can enable alerts through the use of the stats command.

The following is a valid query for alerting on messages that include the text “Exception”:

Copy

filter @message like /Exception/
    | stats count(*) as exceptionCount by bin(1h)
    | sort exceptionCount desc

For more information on Grafana alerts, refer to Alerting.

Deep-link Grafana panels to the CloudWatch console

Left-clicking a time series in the panel displays a context menu with a link to View in CloudWatch console. Clicking the link opens a new tab that takes you to the CloudWatch console and displays all metrics for that query. If you’re not logged in to the CloudWatch console, the link forwards you to the login page. The link provided is valid for any account but displays the expected metrics only if you’re logged in to the account that corresponds to the selected data source in Grafana.

This feature is not available for metrics based on metric math expressions.

Metric Insights queries

The Metrics Query option in the CloudWatch data source is referred to as Metric Insights in the AWS console. It’s a fast, flexible, SQL-based query engine that you can use to identify trends and patterns across millions of operational metrics in real time.

The metrics query editor’s Metrics Query option has two editing modes:

• Builder mode, which provides a visual query-building interface.
• Code mode, which provides a code editor for writing queries.

Use Metric Insights syntax

Metric Insights uses a dialect of SQL and this query syntax:

SQL Copy

SELECT FUNCTION(MetricName)
FROM Namespace | SCHEMA(...)
[ WHERE labelKey OPERATOR labelValue [AND|...]]
[ GROUP BY labelKey [, ...]]
[ ORDER BY FUNCTION() [DESC | ASC] ]
[ LIMIT number]

For details about the Metrics Insights syntax, refer to the AWS reference documentation.

For information about Metrics Insights limits, refer to the AWS feature documentation.

You can also augment queries by using template variables.

Use Metrics Insights keywords

This table summarizes common Metrics Insights query keywords:

Query CloudWatch Logs

The logs query editor helps you write CloudWatch Logs Query Language queries across specified regions and log groups.

It supports querying CloudWatch logs with three query language options:

• Logs Insights QL - The AWS native query language specifically designed for CloudWatch Logs. It uses a SQL-like syntax with commands like fields, filter, stats, and sort. It’s optimized for CloudWatch’s log structure and offers built-in functions for parsing timestamps, extracting fields from JSON logs, and performing aggregations.
• OpenSearch PPL - The OpenSearch query language is based on Elasticsearch’s query DSL (Domain Specific Language). It uses a pipe-based syntax similar to Unix command-line tools or the Splunk search language, and supports complex boolean logic, range queries, wildcard matching, and full-text search capabilities.
• OpenSearch SQL - OpenSearch SQL is a query language that uses a SQL-like syntax for querying data in OpenSearch. It supports standard SQL queries and is designed for users familiar with SQL.

Create a CloudWatch Logs query:

1. Select a region.

2. Select CloudWatch Logs from the query type drop-down.

3. Select the query language you would like to use in the Query Language drop-down.

4. Click Select log groups and choose up to 20 log groups to query.

5. Use the main input area to write your logs query. Amazon CloudWatch only supports a subset of OpenSearch SQL and PPL commands. To find out more about the syntax supported, consult Amazon CloudWatch Logs documentation

   Note

   You must specify the region and log groups when querying with Logs Insights QL and OpenSearch PPL. OpenSearch SQL doesn’t require log group selection. However, selecting log groups simplifies query writing by populating syntax suggestions with discovered log group fields.

Click CloudWatch Logs Insights to interactively view, search, and analyze your log data in the CloudWatch Logs Insights console. If you’re not logged in to the CloudWatch console, the link forwards you to the login page.

Query Log groups with OpenSearch SQL

When querying log groups with OpenSearch SQL, you must explicitly state the log group identifier or ARN in the FROM clause:

SQL Copy

SELECT window.start, COUNT(*) AS exceptionCount
FROM `log_group`
WHERE `@message` LIKE '%Exception%'

or, when querying multiple log groups:

SQL Copy

SELECT window.start, COUNT(*) AS exceptionCount
FROM `logGroups( logGroupIdentifier: ['LogGroup1', 'LogGroup2'])`
WHERE `@message` LIKE '%Exception%'

You can also write queries returning time series data by using the stats command. When making stats queries in Explore, ensure you are in Metrics Explore mode.

Cross-account observability

The CloudWatch plugin monitors and troubleshoots applications that span multiple accounts within a region. Cross-account observability enables seamless searching, visualization, and analysis of metrics and logs across account boundaries.

To enable cross-account observability, complete the following steps:

1. Go to the Amazon CloudWatch documentation and follow the instructions for enabling cross-account observability.

2. Add two API actions to the IAM policy attached to the role/user running the plugin.

Cross-account querying is available in the plugin through the Logs, Metric search, and Metric Insights modes. After you have configured it, you’ll see a Monitoring account badge in the query editor header.

Query caching

When you enable query and resource caching, Grafana temporarily stores the results of data source queries and resource requests. Query caching is available in CloudWatch Metrics in Grafana Cloud and Grafana Enterprise. It is not available in CloudWatch Logs Insights due to how query results are polled from AWS.