Documentationbreadcrumb arrow Pluginsbreadcrumb arrow Amazon Athenabreadcrumb arrow Troubleshooting
Grafana Cloud Enterprise Open source

Troubleshoot Amazon Athena data source issues

This document provides solutions to common issues you may encounter when configuring or using the Amazon Athena data source. For configuration instructions, refer to Configure the Amazon Athena data source.

Authentication errors

These errors occur when AWS credentials are invalid, missing, or don’t have the required permissions.

“Access denied” or “Authorization failed”

Symptoms:

  • Save & test fails with authorization errors.
  • Queries return access denied messages.
  • Resource drop-downs (catalogs, databases, tables) don’t load.

Possible causes and solutions:

CauseSolution
Missing IAM permissionsVerify the IAM user or role has all required Athena, Glue, and S3 permissions. Refer to the IAM policies section for a minimal policy example.
Invalid access keysVerify the access key and secret key in the data source configuration. Regenerate them in the AWS IAM console if necessary.
Expired credentialsCreate new credentials and update the data source configuration.
Wrong regionVerify the Default Region setting matches the region where your Athena resources are located.
Assume role misconfigurationVerify the Assume Role ARN is correct and that the trust policy on the target role allows the source identity to assume it.
External ID mismatchIf the target role requires an external ID, verify the External ID field matches the value configured in the role’s trust policy.
Lake Formation permissionsIf your data is managed by AWS Lake Formation, add lakeformation:GetDataAccess to your IAM policy. Refer to Access denied with AWS Lake Formation.

Access denied with AWS Lake Formation

Symptoms:

  • Save & test succeeds but queries return access denied errors.
  • Queries fail despite having valid Athena, Glue, and S3 permissions.
  • Errors occur only for tables managed by AWS Lake Formation.

Solutions:

  1. Add the lakeformation:GetDataAccess permission to the IAM policy used by the data source. Refer to the IAM policies section for the full policy example.
  2. Verify the IAM identity is registered as a data lake administrator or has been granted table-level permissions in Lake Formation. Refer to the AWS Lake Formation permissions for Athena documentation.

Invalid token errors in private clouds

Symptoms:

  • Authentication fails with invalid token errors.
  • Occurs when using AWS GovCloud or other private cloud partitions.

Solutions:

  1. Ensure you’re using the correct endpoint for your private cloud partition.
  2. Set a custom Endpoint in the data source configuration that matches your private cloud.
  3. Verify the region is set to the correct GovCloud or private partition region.

Connection errors

These errors occur when Grafana can’t reach the Amazon Athena service endpoints.

“Failed to create HTTP client” or connection timeout

Symptoms:

  • Save & test fails with connection errors.
  • Queries time out.
  • Drop-downs fail to load.

Solutions:

  1. Verify network connectivity from the Grafana server to AWS endpoints.
  2. Check that firewall rules allow outbound HTTPS (port 443) to AWS service endpoints.
  3. If you’re using a custom Endpoint, verify the URL is correct and reachable.
  4. For Grafana Cloud accessing private resources, configure Private data source connect.

Query errors

These errors occur when executing queries against Amazon Athena.

“No data” or empty results

Symptoms:

  • The query executes without error but returns no data.
  • Charts display a “No data” message.

Possible causes and solutions:

CauseSolution
Time range doesn’t contain dataExpand the dashboard time range or verify data exists for the selected period in Amazon Athena.
Wrong database or table selectedVerify the correct catalog, database, and table are selected in the resource selectors.
Permissions issueVerify the IAM identity has read access to the S3 location where the table data is stored.
Missing Glue permissionsAthena uses AWS Glue for metadata. Verify Glue permissions are included in your IAM policy.

Query timeout

Symptoms:

  • The query runs for a long time then fails.
  • Error mentions timeout or query limits.

Solutions:

  1. Narrow the dashboard time range to reduce the volume of data scanned.
  2. Add filters to your query to reduce the result set.
  3. Use partitioned tables to limit the amount of data Athena scans.
  4. Break complex queries into smaller parts.
  5. Verify that async query data support is enabled (it is by default), which prevents long-running queries from timing out.

Macro errors

Symptoms:

  • Query fails with errors referencing macros like $__timeFilter, $__timeGroup, or $__dateFilter.
  • Error messages such as “macro $__timeGroup needs time column and interval” or “expected at least one argument”.

Possible causes and solutions:

CauseSolution
Missing required arguments$__timeGroup requires at least a time column and interval (for example, $__timeGroup(time, '5m')). $__timeFilter and $__dateFilter require a column argument.
Invalid interval formatVerify the interval argument uses a valid format such as '1m', '5m', '1h', or '1d'.
Incorrect date format stringWhen using a custom format with $__timeFilter, ensure the format string follows Presto’s Java Date Functions syntax.

“Unknown column type”

Symptoms:

  • Query fails with an error referencing an unsupported column type.

Solutions:

  1. Check the column types in your Athena table and ensure they’re supported.
  2. Use a CAST expression to convert the column to a supported type, such as CAST(my_column AS VARCHAR).

Query doesn’t work when pasted into Amazon Athena

Symptoms:

  • A query works in Grafana but fails when copied directly into the Amazon Athena console.

Solutions:

  1. Grafana macros (like $__timeFilter) aren’t valid Athena SQL. Use the Query Inspector to view the fully interpolated query. Click the Query Inspector button and select the Query tab to copy the expanded SQL.

Template variable errors

These errors occur when using template variables with the Amazon Athena data source.

Variables return no values

Solutions:

  1. Verify the data source connection is working by running Save & test in the data source settings.
  2. Check that parent variables (for cascading variables) have valid selections.
  3. Verify the IAM identity has permissions to query the tables referenced in the variable query.
  4. Ensure the variable query returns results. Test the query directly in the query editor first.

Variables are slow to load

Solutions:

  1. Set variable refresh to On dashboard load instead of On time range change to reduce how often variables re-query.
  2. Simplify variable queries, for example, by using SELECT DISTINCT column FROM table LIMIT 100 to cap results.

Query result reuse errors

These errors relate to the query result reuse feature.

Result reuse isn’t working

Symptoms:

  • Queries execute fresh each time despite enabling result reuse.

Possible causes and solutions:

CauseSolution
Wrong Athena engine versionResult reuse requires Amazon Athena engine version 3. Refer to Changing Athena engine versions for upgrade instructions.
Feature not enabled per queryEnable the Enabled checkbox in the Query result reuse section of the query editor for each query that should reuse results.
Max age too shortIncrease the Max Age in Minutes setting (default: 60 minutes) if cached results expire before they can be reused.

Performance issues

These issues relate to slow queries or API limits.

API throttling

Symptoms:

  • Intermittent errors across multiple panels.
  • Dashboard panels occasionally fail to load.

Solutions:

  1. Reduce the frequency of dashboard auto-refresh.
  2. Reduce the number of panels querying Athena on a single dashboard.
  3. Enable query caching to reduce the number of API calls (available in Grafana Enterprise and Grafana Cloud).
  4. Request a service quota increase from AWS if you consistently hit API rate limits.

Enable debug logging

To capture detailed error information for troubleshooting:

  1. Set the Grafana log level to debug in the configuration file:

    ini 
    [log]
level = debug

  2. Review logs in /var/log/grafana/grafana.log (or your configured log location).

  3. Look for entries related to grafana-athena-datasource that include request and response details.

  4. Reset the log level to info after troubleshooting to avoid excessive log volume.

Get additional help

If you’ve tried the solutions in this document and still encounter issues:

  1. Check the Grafana community forums for similar issues.
  2. Review the Athena data source plugin GitHub issues for known bugs.
  3. Refer to the Amazon Athena documentation for service-specific guidance.
  4. Contact Grafana Support if you’re a Grafana Cloud Pro, Cloud Advanced, or Grafana Enterprise customer.
  5. When reporting issues, include:
    • Grafana version and plugin version.
    • Error messages (redact sensitive information).
    • Steps to reproduce the issue.
    • Relevant configuration (redact credentials).