How it works

Complexity: Moderate | Infrastructure: Pub/Sub + Alloy | Latency: Streaming

You create a log sink in Cloud Logging with filters that select which logs to export. The sink routes them to a Pub/Sub topic. Alloy subscribes to the topic, processes the logs, and forwards them to Grafana Cloud Logs. Sink filters let you reduce volume at the source.

Considerations

Reliable Pub/Sub message delivery
Log sink filtering to reduce volume
Workload identity authentication
Full processing pipeline
Alloy infrastructure to deploy, maintain, and update
Pub/Sub cost, plus setup steps

Documentation

View the GCP logs documentation.

0:00

Read transcript

Audio transcript

For GCP, log collection looks a bit different. There’s no serverless Lambda-equivalent approach here.

The standard architecture uses log sinks, Pub/Sub, and Alloy working together.

Here’s how it flows: you create a log sink in Cloud Logging with filters to select which logs you want. That sink routes logs to a Pub/Sub topic. Pub/Sub is Google’s managed messaging service.

Then Alloy subscribes to that Pub/Sub topic, receives the logs, processes them, and forwards them to Grafana Cloud Logs.

Why this architecture? Pub/Sub provides reliable, scalable message delivery. Log sink filters let you reduce volume at the source. You only have to export what you actually need.

And Alloy gives you the full processing pipeline for transformation and enrichment.

Yes, this involves more infrastructure to set up. But once you configure it, which you can template with Terraform, it scales beautifully.

This is how organizations successfully get GCP logs into Grafana Cloud.

How it works

Considerations

Documentation

Still have questions?

Get every update