How it works
Complexity: Simple | Infrastructure: Serverless | Latency: Event-driven
S3-based logs collected with Lambda
| Log type | Format | Key insights |
|---|---|---|
| ALB Access Logs | Space-delimited | Request latency, targets, status codes |
| CloudFront Logs | Tab-separated | Edge locations, cache hits |
| S3 Access Logs | Space-delimited | Bucket access patterns |
| CloudTrail | JSON | API activity, audit |
Trade-offs
| Pros | Cons |
|---|---|
| Event-driven, automatic | Lambda execution costs |
| Serverless (no infrastructure) | Format parsing required |
| Custom transformation possible | Cold start latency |
Documentation
View the full documentation. Learning path coming soon!
Script
The Lambda agent, also known as lambda-promtail, is your go-to for collecting logs that AWS services write to S3.
Several important services write directly to S3 instead of CloudWatch. ALB access logs give you HTTP request details: latency, status codes, which backend handled the request. CloudFront logs show CDN performance: edge locations, cache hit rates. CloudTrail logs every AWS API call: who did what, when, from where.
Here’s how it works: you deploy a Lambda function that triggers on S3 events. When logs arrive in the bucket, Lambda processes them, parsing the format and adding labels, then forwards them to Loki in Grafana Cloud.
The beauty is serverless simplicity. No EC2 instances to patch, no containers to manage. Lambda scales automatically with your log volume. You can customize the processing logic to fit your needs.
Trade-offs? Lambda has execution costs. You’re paying per invocation and duration. And you’ll need to configure parsing for each log format since ALB logs are space-delimited, CloudFront is tab-separated, and CloudTrail is JSON.