Slide 5 of 10

AWS logs: Firehose

How it works

AWS logs Firehose architecture: CloudWatch Logs to Amazon Data Firehose to Grafana Cloud Logs

Complexity: Moderate | Infrastructure: Managed | Latency: Buffered (60-900s)

You configure a subscription filter on your CloudWatch log group that sends logs to an Amazon Data Firehose stream. The stream batches the logs and delivers them to Grafana Cloud, with built-in retry and error handling. AWS handles the scaling.

CloudWatch logs collected with Firehose

Log typeSourceKey insights
CloudWatch LogsApplications, servicesApplication events, errors
RDS Instance LogsDatabase instancesQuery logs, slow queries
VPC Flow LogsNetwork interfacesNetwork traffic, security

Considerations

  • High-throughput streaming with automatic scaling
  • Automatic retry and error handling
  • Native CloudWatch integration
  • Cost-effective for high log volumes
  • Buffered delivery latency, typically 60-900 seconds
  • Limited in-flight transformation
  • IAM role setup required, plus Amazon Data Firehose usage cost

Documentation

View the AWS Firehose logs documentation.

Script

Amazon Data Firehose handles most AWS log collection. It collects CloudWatch logs from your applications, RDS instance logs, and VPC Flow Logs with high scalability and cost efficiency.

Firehose is a managed streaming service. You configure a subscription filter on your CloudWatch log group that sends logs to Firehose.

Logs flow in, get batched automatically, and deliver to Grafana Cloud with built-in retry and error handling. You don’t manage any compute. AWS handles the scaling.

Cost matters at scale. Firehose keeps costs predictable for high log volumes, which suits production workloads.

One thing to know is latency. Firehose buffers logs before delivery, typically anywhere from 60 to 900 seconds, depending on your configuration.

That’s fine for most use cases. You’re not usually doing real-time alerting on raw log lines.

For CloudWatch logs at any scale, Firehose is straightforward to operate, cost-effective, and scales seamlessly with growing log volumes.

/
Previous Next