Slide 5 of 10

AWS logs - Firehose

How it works

AWS logs Firehose architecture: CloudWatch Logs to Kinesis Firehose to Grafana Cloud Loki

Complexity: Moderate | Infrastructure: Managed | Latency: Buffered (60-900s)

CloudWatch-based logs collected with Firehose

Log typeSourceKey insights
CloudWatch LogsApplications, servicesApplication events, errors
RDS Instance LogsDatabase instancesQuery logs, slow queries
VPC Flow LogsNetwork interfacesNetwork traffic, security

Trade-offs

ProsCons
High-throughput streamingBuffering adds latency.
Managed (automatic scaling)Firehose costs
Automatic retry/error handlingLimited transformation
Native CloudWatch integrationIAM role setup

Documentation

View the full documentation. Learning path coming soon!

AWS Firehose logs

Script

Amazon Data Firehose is your go-to for collecting CloudWatch-based logs. This includes CloudWatch Logs from your applications, RDS instance logs, and VPC Flow Logs.

Firehose is a managed streaming service. You configure a subscription filter on your CloudWatch log group that sends logs to Firehose. Logs flow in, get batched automatically, and deliver to Grafana Cloud with built-in retry and error handling. You don’t manage any compute. AWS handles the scaling.

The trade-off is latency. Firehose buffers logs before delivery, typically anywhere from 60 to 900 seconds depending on your configuration. That’s fine for most use cases. You’re not usually doing real-time alerting on raw log lines.

For CloudWatch-based logs at any scale, Firehose is the recommended approach. It’s simpler to operate, has more predictable costs, and scales seamlessly with growing log volumes.

/
Previous Next