Collection methods by cloud provider
| Cloud | Method | Best for |
|---|---|---|
| AWS | Lambda agent | S3-based logs (ALB, CloudFront, CloudTrail) |
| AWS | Firehose | CloudWatch-based logs (app logs, RDS, VPC Flow) |
| Azure | Functions | Event Hub logs |
| Azure | Alloy | Blob storage, Log Analytics |
| GCP | Alloy + Pub/Sub | All GCP logs |
Quick decision for AWS
| Log source | Recommended method |
|---|---|
| ALB, CloudFront, CloudTrail, S3 access logs | Lambda agent |
| CloudWatch Logs, RDS, VPC Flow Logs | Firehose |
Script
For logs, your collection options look a bit different than metrics.
In AWS, you have two main methods. The Lambda agent, also called lambda-promtail, handles S3-based logs: ALB access logs, CloudFront logs, CloudTrail audit logs. These services write directly to S3 instead of CloudWatch, and Lambda triggers on S3 events to forward them to Loki.
Amazon Data Firehose handles CloudWatch-based logs: your application logs, RDS instance logs, VPC Flow Logs. Firehose is a managed streaming service with built-in scaling and retry handling.
For Azure, you have similar options, Azure Functions for event-driven collection, or Grafana Alloy for more flexibility.
For GCP, there’s no serverless option. The standard approach is to route logs through Pub/Sub and have Alloy subscribe to that topic.
So your choice depends on where your logs actually live and which cloud you’re using. Let’s go through each approach.