Plugins 〉Amazon Aurora


Developer
Grafana

Resources

Sign up to receive occasional product news and updates:



Data Source
grafana
Enterprise

Amazon Aurora

Documentation →

Amazon Aurora Datasource

Query and visualize your data from AWS Aurora with Grafana. This data source plugin is currently in public preview.

Introduction

Using the Amazon Aurora Datasource you can easily query and visualize your data in Amazon Aurora. This plugin currently has support for both Mysql-compatible and Postgres-compatible Aurora Engines.

Screenshot of a timeseries query with aurora

Requirements

This plugin was designed for Enterprise versions of Grafana 9.4.7 and above, although it may work for older versions as well.

This plugin connects to AWS Aurora clusters using IAM with the AWS SDK for go. Please read through the Prerequisites section of the "Connecting to your DB cluster using IAM authentication and the AWS SDK for Go" guide from AWS on how to connect to your db cluster using IAM, to ensure Grafana will be able to query your cluster.

Example permissions

As part of the above, please ensure whatever user/role that you are using to query Aurora has the appropriate permissions. For example:


{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
             "rds-db:connect"
         ],
         "Resource": [
             "arn:aws:rds-db:us-east-2:1234567890:dbuser:cluster-ABCDEFGHIJKL01234/db_user"
         ]
      }
   ]
}

For more information please read through the documentation from AWS

Getting started

  1. Install the plugin
  2. Add a new data source with the UI or provision one
  3. Configure the data source
  4. Start making queries

Configuration options

The Amazon Aurora data source uses the AWS SDK for Go to connect to your DB cluster with AWS IAM Credentials. There are a few possible ways to configure your data source instance with Grafana to specify how you'd like to connect with AWS:

Configuration OptionDetails
Authentication ProviderDepending on the environment in which it is run, Grafana supports different authentication providers such as keys, a credentials file, or using the "Default" provider from AWS which supports using service-based IAM roles. These providers can be manually enabled/disabled with the allowed_auth_providers field. To read more about supported authentication providers please see the Cloud Watch Data Source's documentation
Assume Role ARNAn optional field, to be used if you would like whatever provider you selected above to assume a role.
External IdAn optional field, sometimes used when assuming a role
EndpointAn optional field, by default, all requests will use whatever endpoint is specified in the AWS SDK. However if you would like to override this, you can.
Default RegionSpecify the region of your cluster.
EngineChoose which Aurora Engine your cluster uses. At this time we provide support for PostgreSQL and MySQL compatible engines.
Database Namean optional field, used to create the DSN.
Database UserUsed to create the DSN.
Database HostUsed to create the DSN (note: it is recommended you choose "specify a read-only connection", as there is nothing in the Grafana UI that will prevent writes to your DB).
Database PortUsed to create the DSN.

Advanced Settings

In order to connect to your cluster, Grafana makes 2 calls:

  1. Generating an auth token with RDS with an endpoint
  2. Opening a connection to your DSN with that token and your endpoint.

Often the endpoint for these two steps is the same. However if you have your DB cluster behind a load balancer you may need 2 separate endpoints. For the first step you will need to specify the endpoint behind the load balancer. For the second step you will need to specify the load balancer endpoint for open SQL connections. To have Grafana do this on your behalf, specify the endpoint behind the load balancer in the DB Host for Auth and DB Port for Auth and use the load balancer endpoint/port configuration fields above.

Example of provisioned Aurora data source

If you're provisioning your data source you can set all of the above configuration options with a YAML file. Here's an example of what that file might look like:

  - name: AWS Aurora with mysql behind a load balancer
    type: grafana-aurora-datasource
    editable: true
    jsonData:
      engine: 'aurora-mysql'
      authType: keys
      dbName: 'testDatabase'
      dbUser: 'dbuser'
      dbHost: 'aurora-mysql.cluster-123.us-east-1.rds.amazonaws.com'
      dbPort: 3306
      defaultRegion: 'us-east-1'
    secureJsonData:
      accessKey: someAccessKey
      secretKey: someSecretKey
    version: 1

Here is another example with a load balancer:

  - name: AWS Aurora with mysql behind a load balancer
    type: grafana-aurora-datasource
    editable: true
    jsonData:
      engine: 'aurora-mysql'
      authType: keys
      dbName: 'testDatabase'
      dbUser: 'dbuser'
      dbHost: 'protectedByALoadBalancer.example.com'
      dbPort: 3307
      defaultRegion: 'us-east-1'
      dbPortAuth: 3306
      dbHostAuth 'aurora-mysql.cluster-123.us-east-1.rds.amazonaws.com'
    secureJsonData:
      accessKey: someAccessKey
      secretKey: someSecretKey
    version: 1

Querying

Table vs time series

As with other SQL data source plugins in Grafana, you can specify whether to return data as a "wide" or "long" time series when querying. This is controlled by the "Format as dropdown".

To better explain this concept, consider a table like this:

Using Format As Table with a Table Visualization

When we render this same query with the time series visualization, we'll notice that the "neighborhood" and "city" fields collapse into 1 time series:

Using Format As Table with a Time Series Visualization

This is often the desired and expected behavior from a sql datasource, particularly when using the Table Visualization. However, there are times when you may want to render a separate time series for each city/neighborhood combo. To do so, in the "Format as" dropdown select "Time series" which will render that same timeseries visualization like so:

Using Format as Time Series with a Time Series Visualization

When using Explore, Grafana renders your data using the appropriate time series or table visualization automatically when you change the format. With the query editor in dashboards, users control how to visualize the data.

To learn more, refer to the Grafana documentation time series formats.

Variables and macros

This plugin supports both custom and global variables to make writing queries easier. For example, if you set a custom variable to be $tableName you can use the same query in a dashboard across multiple tables:

select * from $tableName limit 3;

{{< admonition type="note" }} Variables don't work with Grafana Alerting. {{ /admonition >}}

This plugin also supports macros from the sql util library such as $__timeFrom and $__timeTo. For example:

select * from test_table where $__timeFrom(recorded_at) and $__timeTo(recorded_at)

If you have a suggestion for a macro unique to the aurora datasource, feel free to reach out to customer support.

Installing Amazon Aurora on Grafana Cloud:

For more information, visit the docs on plugin installation.

Changelog

0.2.4

  • Incorporate downstream fixes for db connection leak in #65
  • Add dependabot for sdk in #62

0.2.3

  • Add errorsource, update dependencies (#60)
  • Bump grafana-plugin-sdk-go to 0.250.2 (#61)
  • Bump grafana-aws-sdk to 0.31.2 (#61)

0.2.2

  • Fix docs syntax

0.2.1

  • Update plugin.json data source name to follow conventions
  • Update READMEs and docs for style and to follow name conventions
  • Update release process docs

0.2.0

  • Initial public preview release

0.1.11 (Dev release)

  • Use absolute paths for images

0.1.10 (Dev release)

  • Update dependencies

0.1.9 (Dev release)

  • Add docs

0.1.8 (Private Release for test customers)

  • Add support for table vs timeseries format dropdown

0.1.7 (Private Release for test customers)

  • Add support for mysql and fix issue with postgres converters

0.1.6 (Private Release for test customers)

  • Add support for a separate port for generating auth token

0.1.5 (Private Release for test customers)

  • Add support for variables/macros

0.1.4 (Private Release for test customers)

  • Support assume role with a specific endpoint
  • Update plugin-e2e

0.1.3 (Private Release for test customers)

  • Remove log lines

0.1.2 (Private Release for test customers)

  • Adds additional host option for generating auth tokens

0.1.1 (Private Release for test customers)

  • Add support Assuming a role
  • Adds new config editor pages
  • temporarily adds log lines for better debugging

0.1.0 (Private Release for test customers)

Initial beta release.