Plugins 〉Amazon Aurora
Amazon Aurora
Amazon Aurora Datasource
Query and visualize your data from AWS Aurora with Grafana. This data source plugin is currently in public preview.
Introduction
Using the Amazon Aurora Datasource you can easily query and visualize your data in Amazon Aurora. This plugin currently has support for both Mysql-compatible and Postgres-compatible Aurora Engines.
Requirements
This plugin was designed for Enterprise versions of Grafana 9.4.7 and above, although it may work for older versions as well.
This plugin connects to AWS Aurora clusters using IAM with the AWS SDK for go. Please read through the Prerequisites section of the "Connecting to your DB cluster using IAM authentication and the AWS SDK for Go" guide from AWS on how to connect to your db cluster using IAM, to ensure Grafana will be able to query your cluster.
Example permissions
As part of the above, please ensure whatever user/role that you are using to query Aurora has the appropriate permissions. For example:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:us-east-2:1234567890:dbuser:cluster-ABCDEFGHIJKL01234/db_user"
]
}
]
}
For more information please read through the documentation from AWS
Getting started
- Install the plugin
- Add a new data source with the UI or provision one
- Configure the data source
- Start making queries
Configuration options
The Amazon Aurora data source uses the AWS SDK for Go to connect to your DB cluster with AWS IAM Credentials. There are a few possible ways to configure your data source instance with Grafana to specify how you'd like to connect with AWS:
Configuration Option | Details |
---|---|
Authentication Provider | Depending on the environment in which it is run, Grafana supports different authentication providers such as keys, a credentials file, or using the "Default" provider from AWS which supports using service-based IAM roles. These providers can be manually enabled/disabled with the allowed_auth_providers field. To read more about supported authentication providers please see the Cloud Watch Data Source's documentation |
Assume Role ARN | An optional field, to be used if you would like whatever provider you selected above to assume a role. |
External Id | An optional field, sometimes used when assuming a role |
Endpoint | An optional field, by default, all requests will use whatever endpoint is specified in the AWS SDK. However if you would like to override this, you can. |
Default Region | Specify the region of your cluster. |
Engine | Choose which Aurora Engine your cluster uses. At this time we provide support for PostgreSQL and MySQL compatible engines. |
Database Name | an optional field, used to create the DSN. |
Database User | Used to create the DSN. |
Database Host | Used to create the DSN (note: it is recommended you choose "specify a read-only connection", as there is nothing in the Grafana UI that will prevent writes to your DB). |
Database Port | Used to create the DSN. |
Advanced Settings
In order to connect to your cluster, Grafana makes 2 calls:
- Generating an auth token with RDS with an endpoint
- Opening a connection to your DSN with that token and your endpoint.
Often the endpoint for these two steps is the same. However if you have your DB cluster behind a load balancer you may need 2 separate endpoints. For the first step you will need to specify the endpoint behind the load balancer. For the second step you will need to specify the load balancer endpoint for open SQL connections. To have Grafana do this on your behalf, specify the endpoint behind the load balancer in the DB Host for Auth
and DB Port for Auth
and use the load balancer endpoint/port configuration fields above.
Example of provisioned Aurora data source
If you're provisioning your data source you can set all of the above configuration options with a YAML file. Here's an example of what that file might look like:
- name: AWS Aurora with mysql behind a load balancer
type: grafana-aurora-datasource
editable: true
jsonData:
engine: 'aurora-mysql'
authType: keys
dbName: 'testDatabase'
dbUser: 'dbuser'
dbHost: 'aurora-mysql.cluster-123.us-east-1.rds.amazonaws.com'
dbPort: 3306
defaultRegion: 'us-east-1'
secureJsonData:
accessKey: someAccessKey
secretKey: someSecretKey
version: 1
Here is another example with a load balancer:
- name: AWS Aurora with mysql behind a load balancer
type: grafana-aurora-datasource
editable: true
jsonData:
engine: 'aurora-mysql'
authType: keys
dbName: 'testDatabase'
dbUser: 'dbuser'
dbHost: 'protectedByALoadBalancer.example.com'
dbPort: 3307
defaultRegion: 'us-east-1'
dbPortAuth: 3306
dbHostAuth 'aurora-mysql.cluster-123.us-east-1.rds.amazonaws.com'
secureJsonData:
accessKey: someAccessKey
secretKey: someSecretKey
version: 1
Querying
Table vs time series
As with other SQL data source plugins in Grafana, you can specify whether to return data as a "wide" or "long" time series when querying. This is controlled by the "Format as dropdown".
To better explain this concept, consider a table like this:
When we render this same query with the time series visualization, we'll notice that the "neighborhood" and "city" fields collapse into 1 time series:
This is often the desired and expected behavior from a sql datasource, particularly when using the Table Visualization. However, there are times when you may want to render a separate time series for each city/neighborhood combo. To do so, in the "Format as" dropdown select "Time series" which will render that same timeseries visualization like so:
When using Explore, Grafana renders your data using the appropriate time series or table visualization automatically when you change the format. With the query editor in dashboards, users control how to visualize the data.
To learn more, refer to the Grafana documentation time series formats.
Variables and macros
This plugin supports both custom and global variables to make writing queries easier. For example, if you set a custom variable to be $tableName
you can use the same query in a dashboard across multiple tables:
select * from $tableName limit 3;
{{< admonition type="note" }} Variables don't work with Grafana Alerting. {{ /admonition >}}
This plugin also supports macros from the sql util library such as $__timeFrom
and $__timeTo
. For example:
select * from test_table where $__timeFrom(recorded_at) and $__timeTo(recorded_at)
If you have a suggestion for a macro unique to the aurora datasource, feel free to reach out to customer support.
Grafana Cloud Free
- Free tier: Limited to 3 users
- Paid plans: $55 / user / month above included usage
- Access to all Enterprise Plugins
- Fully managed service (not available to self-manage)
Self-hosted Grafana Enterprise
- Access to all Enterprise plugins
- All Grafana Enterprise features
- Self-manage on your own infrastructure
Grafana Cloud Free
- Free tier: Limited to 3 users
- Paid plans: $55 / user / month above included usage
- Access to all Enterprise Plugins
- Fully managed service (not available to self-manage)
Self-hosted Grafana Enterprise
- Access to all Enterprise plugins
- All Grafana Enterprise features
- Self-manage on your own infrastructure
Grafana Cloud Free
- Free tier: Limited to 3 users
- Paid plans: $55 / user / month above included usage
- Access to all Enterprise Plugins
- Fully managed service (not available to self-manage)
Self-hosted Grafana Enterprise
- Access to all Enterprise plugins
- All Grafana Enterprise features
- Self-manage on your own infrastructure
Grafana Cloud Free
- Free tier: Limited to 3 users
- Paid plans: $55 / user / month above included usage
- Access to all Enterprise Plugins
- Fully managed service (not available to self-manage)
Self-hosted Grafana Enterprise
- Access to all Enterprise plugins
- All Grafana Enterprise features
- Self-manage on your own infrastructure
Grafana Cloud Free
- Free tier: Limited to 3 users
- Paid plans: $55 / user / month above included usage
- Access to all Enterprise Plugins
- Fully managed service (not available to self-manage)
Self-hosted Grafana Enterprise
- Access to all Enterprise plugins
- All Grafana Enterprise features
- Self-manage on your own infrastructure
Installing Amazon Aurora on Grafana Cloud:
Installing plugins on a Grafana Cloud instance is a one-click install; same with updates. Cool, right?
Note that it could take up to 1 minute to see the plugin show up in your Grafana.
Installing plugins on a Grafana Cloud instance is a one-click install; same with updates. Cool, right?
Note that it could take up to 1 minute to see the plugin show up in your Grafana.
Installing plugins on a Grafana Cloud instance is a one-click install; same with updates. Cool, right?
Note that it could take up to 1 minute to see the plugin show up in your Grafana.
Installing plugins on a Grafana Cloud instance is a one-click install; same with updates. Cool, right?
Note that it could take up to 1 minute to see the plugin show up in your Grafana.
Installing plugins on a Grafana Cloud instance is a one-click install; same with updates. Cool, right?
Note that it could take up to 1 minute to see the plugin show up in your Grafana.
Installing plugins on a Grafana Cloud instance is a one-click install; same with updates. Cool, right?
Note that it could take up to 1 minute to see the plugin show up in your Grafana.
Installing plugins on a Grafana Cloud instance is a one-click install; same with updates. Cool, right?
Note that it could take up to 1 minute to see the plugin show up in your Grafana.
For more information, visit the docs on plugin installation.
Installing on a local Grafana:
For local instances, plugins are installed and updated via a simple CLI command. Plugins are not updated automatically, however you will be notified when updates are available right within your Grafana.
1. Install the Data Source
Use the grafana-cli tool to install Amazon Aurora from the commandline:
grafana-cli plugins install
The plugin will be installed into your grafana plugins directory; the default is /var/lib/grafana/plugins. More information on the cli tool.
Alternatively, you can manually download the .zip file for your architecture below and unpack it into your grafana plugins directory.
Alternatively, you can manually download the .zip file and unpack it into your grafana plugins directory.
2. Configure the Data Source
Accessed from the Grafana main menu, newly installed data sources can be added immediately within the Data Sources section.
Next, click the Add data source button in the upper right. The data source will be available for selection in the Type select box.
To see a list of installed data sources, click the Plugins item in the main menu. Both core data sources and installed data sources will appear.
Changelog
0.2.4
0.2.3
- Add errorsource, update dependencies (#60)
- Bump grafana-plugin-sdk-go to 0.250.2 (#61)
- Bump grafana-aws-sdk to 0.31.2 (#61)
0.2.2
- Fix docs syntax
0.2.1
- Update plugin.json data source name to follow conventions
- Update READMEs and docs for style and to follow name conventions
- Update release process docs
0.2.0
- Initial public preview release
0.1.11 (Dev release)
- Use absolute paths for images
0.1.10 (Dev release)
- Update dependencies
0.1.9 (Dev release)
- Add docs
0.1.8 (Private Release for test customers)
- Add support for table vs timeseries format dropdown
0.1.7 (Private Release for test customers)
- Add support for mysql and fix issue with postgres converters
0.1.6 (Private Release for test customers)
- Add support for a separate port for generating auth token
0.1.5 (Private Release for test customers)
- Add support for variables/macros
0.1.4 (Private Release for test customers)
- Support assume role with a specific endpoint
- Update plugin-e2e
0.1.3 (Private Release for test customers)
- Remove log lines
0.1.2 (Private Release for test customers)
- Adds additional host option for generating auth tokens
0.1.1 (Private Release for test customers)
- Add support Assuming a role
- Adds new config editor pages
- temporarily adds log lines for better debugging
0.1.0 (Private Release for test customers)
Initial beta release.