Collection methods by cloud provider
| Cloud | Method | Best for |
|---|---|---|
| AWS | Firehose | High-volume CloudWatch logs (most use cases) |
| AWS | Lambda Promtail | Specific log types (ALB, CloudFront, CloudTrail); can filter logs before forwarding |
| Azure | Functions | Event-driven from Event Hub |
| Azure | Alloy | Event Hub with flexible processing |
| GCP | Alloy + Pub/Sub | All GCP logs |
Script
For logs, your collection options look a bit different than metrics.
In AWS, you have two main methods. Amazon Data Firehose is a managed streaming service for most CloudWatch logs. It scales to high log volumes seamlessly, with built-in retry handling.
Lambda Promtail is for specific log types that Firehose doesn’t support, like ALB access logs, CloudFront logs, and CloudTrail audit logs. It triggers on events to process and forward these logs to Grafana Cloud Logs, and it can also filter logs before forwarding them.
For Azure, you have similar options: Azure Functions for event-driven collection from Event Hub, or Grafana Alloy for more flexibility.
For GCP, there’s no serverless option. The standard approach is to route logs through Pub/Sub and have Alloy subscribe to that topic.
So your choice depends on your log volumes, cost considerations, and which specific log types you need. Let’s go through each approach.