Configure AWS Monitoring with Terraform

You can define as code AWS monitoring scrape jobs and accounts using Terraform.

Before you begin, you must have the following installed:

cURL
jq

Grant Grafana Cloud Access to Your CloudWatch Data

This sections will help you use Terraform to configure an IAM role and access policy in your AWS Account, which will authorize Grafana Cloud to access your CloudWatch data.

Before you begin

Click Details in the Prometheus card of the Grafana Cloud Portal to find:

The username / instance ID for your Grafana Cloud Prometheus. You can also find this information from the grafana_cloud_stack data source.
The Terraform snippet you need to provision the IAM role

Input variables

The input variables for the IAM role are:

external_id: The username / instance ID for your Grafana Cloud Prometheus. AWS uses an external ID to provide an extra layer of security when giving Grafana access to pull your CloudWatch metrics into Grafana Cloud.
iam_role_name: A customizable name of the IAM role used by Grafana for the CloudWatch integration. The default value is GrafanaCloudWatchIntegration.

Output variable

The output variable is role_arn, which is the IAM role ARN you need to use when you create the scrape job.

Create a new AWS role

Create an AWS role so that Grafana can then assume a role that has access only to your CloudWatch data, with no need to share access and secret keys.

At the Create new scrape job configuration page, select Automatically to create a new role in the AWS IAM console.
Click Use Terraform.
Configure the AWS CLI.

Copy this snippet into your Terraform file.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}
locals {
  grafana_account_id = "008923505280"
}
variable "external_id" {
  type        = string
  description = "This is your Grafana Cloud identifier and is used for security purposes."
  validation {
    condition     = length(var.external_id) > 0
    error_message = "ExternalID is required."
  }
}
variable "iam_role_name" {
  type        = string
  default     = "GrafanaLabsCloudWatchIntegration"
  description = "Customize the name of the IAM role used by Grafana for the CloudWatch integration."
}
data "aws_iam_policy_document" "trust_grafana" {
  statement {
    effect = "Allow"
    principals {
      type        = "AWS"
      identifiers = ["arn:aws:iam::${local.grafana_account_id}:root"]
    }
    actions = ["sts:AssumeRole"]
    condition {
      test     = "StringEquals"
      variable = "sts:ExternalId"
      values   = [var.external_id]
    }
  }
}
resource "aws_iam_role" "grafana_labs_cloudwatch_integration" {
  name        = var.iam_role_name
  description = "Role used by Grafana CloudWatch integration."
  # Allow Grafana Labs' AWS account to assume this role.
  assume_role_policy = data.aws_iam_policy_document.trust_grafana.json

  # This policy allows the role to discover metrics via tags and export them.
  inline_policy {
    name = var.iam_role_name
    policy = jsonencode({
      Version = "2012-10-17"
      Statement = [
        {
          Effect = "Allow"
          Action = [
            "tag:GetResources",
            "cloudwatch:GetMetricData",
            "cloudwatch:ListMetrics",
            "apigateway:GET",
            "aps:ListWorkspaces",
            "autoscaling:DescribeAutoScalingGroups",
            "dms:DescribeReplicationInstances",
            "dms:DescribeReplicationTasks",
            "ec2:DescribeTransitGatewayAttachments",
            "ec2:DescribeSpotFleetRequests",
            "shield:ListProtections",
            "storagegateway:ListGateways",
            "storagegateway:ListTagsForResource"
          ]
          Resource = "*"
        }
      ]
    })
  }
}
output "role_arn" {
  value       = aws_iam_role.grafana_labs_cloudwatch_integration.arn
  description = "The ARN for the role created, copy this into Grafana Cloud installation."
}

Run the terraform apply command, and either set variables directly in the CLI or create a tfvars file as the following shows:
- To set the variables directly in the CLI, use the following example: bash terraform apply -var="grafana_importer_external_id=<your external ID>" -var="iam_role_name=GrafanaCloudWatchIntegration"
- To create a tfvars file (.tfvars), add the following text:
  terraform
```
grafana_importer_external_id="<your external ID>"
iam_role_name="GrafanaCloudWatchIntegration"
```
  Run the following command:
  bash
```
terraform apply -var-file="<your-tfvars-file>.tfvars"
```
After the Terraform apply command has finished creating the IAM Role, it outputs your role_arn. For example:
bash
```
role_arn = "arn:aws:iam::<yourAWSAccountID>:role/<iam_role_name>"
```

Configure Grafana Cloud authentication

Configure authentication before using the Grafana Terraform provider.

Create a Grafana Cloud access policy

To create an access policy for your organization in the Grafana Cloud portal, refer to these steps.

Add scopes to the Grafana Cloud access policy

Add scopes to the access policy.

Enter a scope in the Add Scope textbox to search for a scope.
Assign the following required permissions (scopes):
- integration-management:read
- integration-management:write
- stacks:read

Create a Grafana Cloud access policy token

In your Grafana Cloud stack, generate a token to authenticate the provider with the cloud provider API. Follow the on-screen instructions, or refer to the steps in Create one or more access policy tokens.

Obtain the regional Cloud Provider API endpoint

You need the URL for the cloud provider provider API to communicate.

Copy and use the following script to return a list of all the Grafana Cloud stacks you own, along with their respective cloud provider API hostnames.

 curl -sH "Authorization: Bearer <Access Token from previous step>" "https://grafana.com/api/instances" | \
 jq '[.items[]|{stackName: .slug, clusterName:.clusterSlug, cloudProviderAPIURL: "https://cloud-provider-api-\(.clusterSlug).grafana.net"}]'

Select the hostname for the stack you wish to manage. In the following example, the correct hostname for the herokublogpost stack is https://cloud-provider-api-prod-us-central-0.grafana.net.
```
 [
 {
     "stackName": "herokublogpost",
     "clusterName": "prod-us-central-0",
     "cloudProviderAPIURL": "https://cloud-provider-api-prod-us-central-0.grafana.net"
 }
 ]
```

Set up the Grafana Terraform Provider

Set up the Grafana Terraform provider with either of these methods:

Terraform commands
Environmental variables

Configure the Grafana Terraform Provider

Include the provider as a dependency in your Terraform configuration.

 terraform {
   required_providers {
     grafana = {
       source  = "grafana/grafana"
       version = ">= 3.13.1" # minimum required version that includes Cloud Provider support
     }
   }
 }

Configure AWS support for the Grafana Terraform provider with the following snippet, which uses the access token and cloud provider API URL obtained in the previous steps.
```
 provider "grafana" {
   // ...
   cloud_provider_url = <Cloud Provider API URL from previous step>
   cloud_provider_access_token = <Access Token from previous step>
 }
```

Environment variables

When running Terraform commands, set the cloud provider URL and access token in an empty Grafana provider block with environment variables (GRAFANA_CLOUD_PROVIDER_ACCESS_TOKEN and GRAFANA_CLOUD_PROVIDER_URL).

 provider "grafana" {}

Grafana Terraform provider

You can define the following resources and data sources with the Grafana Terraform provider.

Resource descriptions

Resource name	Description	Documentation reference
grafana_cloud_provider_aws_account	Represents an AWS IAM role that authorizes Grafana Cloud to pull AWS CloudWatch metrics for a set of regions. Usually, there’s one of these resources per configured AWS account.	Doc
grafana_cloud_provider_aws_cloudwatch_scrape_job	Represents a Grafana AWS scrape job. This configures Grafana to fetch a list of metrics/statistics for one or many AWS services, and for a given grafana_cloud_provider_aws_account.	Doc

Example Terraform

The following snippets are for obtaining EC2 and RDS metrics. Replace the values with your values.

variable "local_token" {
  type        = string
  sensitive = true
}

terraform {
  required_providers {
    grafana = {
      source  = "grafana/grafana"
      version >= "3.13.1"
    }
  }
}

provider "grafana" {
  # this token is used for calling the grafana.com API, and easily fetching the Grafana instance Stack ID
  cloud_access_policy_token = var.local_token # This token can be the same one created in the sections above
}

provider "aws" {
  region = "us-east-2"
  profile =  "name of the profile for accessing AWS APIs"
}

data "grafana_cloud_stack" "thestack" {
  slug = "name of the stack"
}

resource "grafana_cloud_provider_aws_account" "myaccount" {
  stack_id = data.grafana_cloud_stack.thestack.id
  role_arn = "role arn"
  regions = [
    "us-east-1",
    "us-east-2",
  ]
}

resource "grafana_cloud_provider_aws_cloudwatch_scrape_job" "myaccount-ec2" {
  stack_id                = data.grafana_cloud_stack.thestack.id
  name                    = "tf-managed-scrape-job"
  aws_account_resource_id = grafana_cloud_provider_aws_account.myaccount.resource_id

  service {
    name = "AWS/EC2"

    metric {
      name        = "CPUUtilization"
      statistics  = ["Average"]
    }

    metric {
      name        = "StatusCheckFailed"
      statistics  = ["Maximum"]
    }

    scrape_interval_seconds = 300
    tags_to_add_to_metrics = ["eks:cluster-name"]
  }

  service {
    name = "AWS/RDS"

    metric {
      name        = "CPUUtilization"
      statistics  = ["Average", "Maximum"]
    }

    scrape_interval_seconds = 300
    tags_to_add_to_metrics = ["name"]
  }

  custom_namespace {
    name = "MyApp"

    metric {
      name       = "MyMetric"
      statistics = ["Maximum", "Sum"]
    }

    scrape_interval_seconds = 300
  }
}