---
title: "Configure CloudWatch logs with Lambda | Grafana Cloud documentation"
description: "Configure Logs with Lambda to ship Amazon CloudWatch Logs to Grafana Cloud using CloudFormation or Terraform."
---

# Configure Amazon CloudWatch logs with Lambda

Send logs from Amazon CloudWatch Logs to AWS observability in Grafana Cloud Provider using a [Lambda Promtail](https://github.com/grafana/lambda-promtail) function and a **subscription filter** that invokes the Lambda function when new log events are written.

For supported log types and a consolidated **label reference**, refer to [Logs with Lambda](/docs/grafana-cloud/monitor-infrastructure/monitor-cloud-provider/aws/logs/cloudwatch-logs/). For advanced options (Amazon Kinesis, relabeling, Amazon ECR images), refer to the [Lambda Promtail client](/docs/loki/latest/send-data/lambda-promtail/) documentation.

## Before you begin

Before configuring AWS logs with Lambda Promtail, make sure you have:

- A [Grafana Cloud account](/auth/sign-up/create-user) with a Grafana Cloud stack.
- The [Cloud Provider AWS Writer app plugin role](/docs/grafana-cloud/monitor-infrastructure/monitor-cloud-provider/rbac/#cloud-provider-app-plugin-roles) or [Grafana Cloud Admin basic role](/docs/grafana-cloud/monitor-infrastructure/monitor-cloud-provider/rbac/#grafana-cloud-basic-roles).
- An AWS account with permissions to create Lambda functions, IAM roles, and either CloudWatch subscription filters or S3 notifications in the **same AWS Region** as your log source.
- The [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) configured.
- The Grafana Cloud **Configuration details** values for Loki (Grafana Cloud Loki write address and username, and a Grafana Cloud access policy token), available on the **Logs with Lambda** page in Grafana Cloud Provider.

## Navigate to Logs with Lambda

Before you can begin configuring AWS logs with Lambda Promtail, you must navigate to the Logs with Lambda page in Grafana Cloud Provider.

To navigate to the Logs with Lambda page:

1. Open your [Grafana Cloud stack](/docs/grafana-cloud/security-and-account-management/cloud-stacks/).
2. Expand **Observability > Cloud provider > AWS** in the main menu of your Grafana Cloud stack.
3. Click the **Configuration** tab.
4. Click the **Logs with Lambda** tile.

Then continue with [CloudFormation](#configure-with-cloudformation) or [Terraform](#configure-with-terraform).

## Configure with CloudFormation

The provided CloudFormation template can get you started quickly and requires little additional configuration. CloudFormation is a great option for AWS-native infrastructure definitions.

### Upload Lambda Promtail

Before launching the CloudFormation stack, you need to upload the Lambda Promtail compressed binary file to the AWS S3 bucket in the same region as the AWS S3 bucket with your CloudWatch log groups.

To upload the Lambda Promtail compressed binary to your S3 bucket:

1. On the **Logs with Lambda** page in Grafana Cloud Provider, click **Use CloudFormation**.
2. Copy the upload command from the Grafana Cloud UI.
3. Update the command with YOUR-BUCKET-NAME and YOUR-REGION-NAME and run it in your terminal.
   
   This command uploads the compressed lambda-promtail build to an S3 bucket in the **same AWS Region** where the Lambda function runs and where your CloudWatch log groups are stored.

### Create a Grafana Cloud access policy token

Create a Grafana Cloud access policy token with the **MetricsPublisher** role so Lambda Promtail can authenticate to Grafana Cloud.

To generate a Grafana Cloud access policy token with `MetricsPublisher` role:

1. On the **Logs with Lambda** page in Grafana Cloud Provider, in the **API token name** field, enter a name for the token.
2. Select an Expiration date.
   
   We recommend you select a specific expiration date and do not set the Expiration date to **No expiry**, as this can create a security vulnerability.
3. Click **Create token**.
4. Copy the generated key and store it securely.

### Launch the CloudFormation stack

After you have the Grafana Cloud access policy token, you must launch and customize the CloudFormation stack template. The AWS account you are logged into is the account where resources are created. To use a different account, log out and log into the desired account before launching the CloudFormation stack.

To launch and customize the CloudFormation stack:

1. On the **Logs with Lambda** page in Grafana Cloud Provider, click **Launch CloudFormation stack**.
2. In the AWS CloudFormation console, paste the Grafana Cloud access policy token into the **Password** parameter (or the field your template uses for Loki basic auth).
3. Configure the **LogGroup** and **SubscriptionFilter** parameters so the filter targets the log groups you want to ship.

## Configure with Terraform

The provided Terraform sample code snippets can get you started quickly and require little additional configuration.

To configure logs with Lambda using Terraform, on the **Logs with Lambda** page in Grafana Cloud Provider click **Use Terraform**.

### Create a Grafana Cloud access policy token

Create a Grafana Cloud access policy token with the **MetricsPublisher** role so Lambda Promtail can authenticate to Grafana Cloud.

To generate a Grafana Cloud access policy token with `MetricsPublisher` role:

1. On the **Logs with Lambda** page in Grafana Cloud Provider, in the **API token name** field, enter a name for the token.
2. Select an Expiration date.
   
   We recommend you select a specific expiration date and do not set the Expiration date to **No expiry**, as this can create a security vulnerability.
3. Click **Create token**.
4. Copy the generated key and store it securely.

### Terraform setup

Configure the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html) for the Region where Lambda, your log groups, and the S3 bucket for the zip artifact live.

1. Copy the following Terraform snippet into a `main.tf` file (or merge into your existing root module):
   
   hcl ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy
   
   ```hcl
   terraform {
     required_providers {
       aws = {
         source  = "hashicorp/aws"
         version = "~> 5.0"
       }
     }
   }
   
   data "aws_region" "current" {}
   
   resource "aws_s3_object_copy" "lambda_promtail_zipfile" {
     bucket = var.s3_bucket
     key    = var.s3_key
     source = "grafanalabs-cf-templates/lambda-promtail/lambda-promtail.zip"
   }
   
   resource "aws_iam_role" "lambda_promtail_role" {
     name = "GrafanaLabsCloudWatchLogsIntegration"
   
     assume_role_policy = jsonencode({
       "Version" : "2012-10-17",
       "Statement" : [
         {
           "Action" : "sts:AssumeRole",
           "Principal" : {
             "Service" : "lambda.amazonaws.com"
           },
           "Effect" : "Allow"
         }
       ]
     })
   }
   
   resource "aws_iam_role_policy" "lambda_promtail_policy_logs" {
     name = "lambda-logs"
     role = aws_iam_role.lambda_promtail_role.name
     policy = jsonencode({
       "Statement" : [
         {
           "Action" : [
             "logs:CreateLogGroup",
             "logs:CreateLogStream",
             "logs:PutLogEvents"
           ],
           "Effect" : "Allow",
           "Resource" : "arn:aws:logs:*:*:*"
         }
       ]
     })
   }
   
   resource "aws_cloudwatch_log_group" "lambda_promtail_log_group" {
     name              = "/aws/lambda/GrafanaCloudLambdaPromtail"
     retention_in_days = 14
   }
   
   resource "aws_lambda_function" "lambda_promtail" {
     function_name = "GrafanaCloudLambdaPromtail"
     role          = aws_iam_role.lambda_promtail_role.arn
   
     timeout     = 60
     memory_size = 128
   
     handler   = "main"
     runtime   = "provided.al2023"
     s3_bucket = var.s3_bucket
     s3_key    = var.s3_key
   
     environment {
       variables = {
         WRITE_ADDRESS = var.write_address
         USERNAME      = var.username
         PASSWORD      = var.password
         KEEP_STREAM   = var.keep_stream
         BATCH_SIZE    = var.batch_size
         EXTRA_LABELS  = var.extra_labels
       }
     }
   
     depends_on = [
       aws_s3_object_copy.lambda_promtail_zipfile,
       aws_iam_role_policy.lambda_promtail_policy_logs,
       aws_cloudwatch_log_group.lambda_promtail_log_group
     ]
   }
   
   resource "aws_lambda_function_event_invoke_config" "lambda_promtail_invoke_config" {
     function_name          = aws_lambda_function.lambda_promtail.function_name
     maximum_retry_attempts = 2
   }
   
   resource "aws_lambda_permission" "lambda_promtail_allow_cloudwatch" {
     statement_id  = "lambda-promtail-allow-cloudwatch"
     action        = "lambda:InvokeFunction"
     function_name = aws_lambda_function.lambda_promtail.function_name
     principal     = "logs.${data.aws_region.current.name}.amazonaws.com"
   }
   
   # Subscribe each log group in var.log_group_names. For a different filter_pattern per group,
   # duplicate this resource and set filter_pattern explicitly.
   resource "aws_cloudwatch_log_subscription_filter" "lambda_promtail_logfilter" {
     for_each        = toset(var.log_group_names)
     name            = "lambda_promtail_logfilter_${each.value}"
     log_group_name  = each.value
     destination_arn = aws_lambda_function.lambda_promtail.arn
     filter_pattern  = ""
     depends_on      = [aws_iam_role_policy.lambda_promtail_policy_logs]
   }
   
   output "lambda_arn" {
     value       = aws_lambda_function.lambda_promtail.arn
     description = "ARN of the Lambda function that runs lambda-promtail."
   }
   ```
2. Copy the following Terraform snippet into `variables.tf`:
   
   hcl ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy
   
   ```hcl
   variable "write_address" {
     type        = string
     description = "Grafana Cloud Loki push URL."
     default     = ""
   }
   
   variable "username" {
     type        = string
     description = "Basic auth username for Grafana Cloud Loki."
     default     = ""
   }
   
   variable "password" {
     type        = string
     description = "Basic auth password for Grafana Cloud Loki (Grafana.com API key)."
     sensitive   = true
     default     = ""
   }
   
   variable "s3_bucket" {
     type        = string
     description = "Bucket that holds lambda-promtail.zip."
     default     = ""
   }
   
   variable "s3_key" {
     type        = string
     description = "Object key for lambda-promtail.zip."
     default     = "lambda-promtail.zip"
   }
   
   variable "log_group_names" {
     type        = list(string)
     description = "CloudWatch log group names to subscribe (for example /aws/lambda/my-function)."
     default     = []
   }
   
   variable "keep_stream" {
     type        = string
     description = "If true, keep the CloudWatch log stream name as a Loki label (can increase cardinality)."
     default     = "false"
   }
   
   variable "extra_labels" {
     type        = string
     description = "Comma-separated pairs: name1,value1,name2,value2,..."
     default     = ""
   }
   
   variable "batch_size" {
     type        = string
     description = "Flush threshold in bytes."
     default     = ""
   }
   ```
3. On the **Logs with Lambda** page in Grafana Cloud Provider, copy the **Grafana Cloud Loki write address**, and **Grafana Cloud Loki username**, and your Grafana Cloud access policy token into a **Terraform variables file** (for example `grafana.auto.tfvars`).
   
   > Warning
   > 
   > Do not commit secrets to version control.
   
   hcl ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy
   
   ```hcl
   write_address = "@@@https://logs-prod-...grafana.net/loki/api/v1/push@@@" // Grafana Cloud Loki write address
   username      = "@@@LOKI_USERNAME@@@" // Grafana Cloud Loki username
   password      = "@@@ACCESS_POLICY_TOKEN@@@" // Grafana Cloud access policy token
   s3_bucket     = "@@@my-lambda-artifacts@@@"
   log_group_names = ["@@@/aws/lambda/my-app@@@"]
   ```
4. Update `log_group_names` and `s3_bucket` to the correct bucket names.
5. Initialize and apply using the following command:
   
   Bash ![Copy code to clipboard](/media/images/icons/icon-copy-small-2.svg) Copy
   
   ```bash
   terraform init
   terraform apply
   ```

The snippets above are a minimal starting point. For VPC subnets, security groups, and additional log groups, extend your module or start from the [example main.tf](https://github.com/grafana/lambda-promtail/blob/main/main.tf) in the Lambda Promtail repository.

## Explore logs

After you have connected your AWS logs to Grafana Cloud using Lambda Promtail, you can start exploring.

To explore your logs in Grafana Cloud:

1. On the **Logs with Lambda** page in Grafana Cloud Provider, click **Go to Explore**.
2. In **Explore**, run a query that selects on `__aws_cloudwatch_log_group` to confirm events are arriving.

## Labels for this workflow

CloudWatch log lines include the following labels: `__aws_cloudwatch_log_group`, `__aws_cloudwatch_owner`, and optionally `__aws_cloudwatch_log_stream` when `KEEP_STREAM` is `true`. For the full table and **extra labels** syntax, refer to [Labels in Logs with Lambda](/docs/grafana-cloud/monitor-infrastructure/monitor-cloud-provider/aws/logs/cloudwatch-logs/#labels).