Introducing an OpenTelemetry Collector distribution with built-in Prometheus pipelines: Grafana Alloy
Note: Existing users of Grafana Agent Flow will feel right at home with Alloy because Alloy uses the same components, code, and concepts that were first introduced in Grafana Agent Flow. Read more in our Agent to Alloy blog post.
In the opening keynote of GrafanaCON 2024, we announced our newest OSS project: Grafana Alloy, our open source distribution of the OpenTelemetry Collector. Alloy is a telemetry collector that is 100% OTLP compatible and offers native pipelines for OpenTelemetry and Prometheus telemetry formats, supporting metrics, logs, traces, and profiles.
Some of you may be thinking: Wait, another collector?
Hear us out: What makes Alloy stand out among the growing ecosystem of collectors is that it’s an open source tool that combines all the observability signals that are vital to run combined infrastructure and application monitoring workloads at scale.
Alloy combines the decade of industry-leading observability codebase of Grafana Agent with the lessons learned from some of the toughest use cases we’ve seen over the years. As a result, Alloy is both a respectfully opinionated OpenTelemetry Collector distribution and the most efficient and cost-effective way to do Prometheus-compatible metrics. Plus, it includes enterprise-grade features — such as native clustering for production at scale and built-in Vault support for enhanced security — all out of the box.
Overall, Alloy is an extension of our “big tent” philosophy at Grafana Labs, where disparate data sources, from different software providers, in different industries, built for different use cases, can come together in a composable observability platform. We believe that organizations should own their observability strategy, choose their own tools, and have the freedom to bring all their data together in one view. Grafana Alloy helps you achieve just that as a telemetry collector that’s compatible with a suite of different signals that can meet a wide array of different needs.
Let’s take a deeper look at the features offered in Grafana Alloy — and the continued innovation that you can expect from our latest OSS project.
From Agent to Alloy
Grafana Agent, our existing OSS telemetry agent, and Grafana Agent Operator are now deprecated and will enter into Long-Term Support (LTS) beginning today (April 9, 2024). This means both Grafana Agent and Grafana Agent Operator will no longer receive any new feature updates, but they will receive critical bug fixes and security fixes. Commercial support will end after the LTS phase, which we anticipate will extend for about 18 months until October 31, 2025. End-of-Life (EOL) phase for Grafana Agent and Grafana Agent Operator will begin once LTS ends. Grafana Agent and the Grafana Agent Operator are expected to reach EOL on November 1, 2025.
We recommend that you begin planning your transition towards the new Grafana Alloy collector. To learn more, check out our Agent to Alloy FAQ blog post.
OpenTelemetry and Prometheus telemetry with Alloy
According to Grafana Labs’ 2024 Observability Survey, an overwhelming majority of respondents report that they are investing in Prometheus (89%) or OpenTelemetry (85%). Almost 40% of respondents use both in production, and more than 50% increased their usage of both projects over the past year.
And it makes sense why. Prometheus is still the most mature ecosystem for core infrastructure monitoring use cases, with more than 10,000 exporters and integrations in the wild that have been built by the open source community over the past decade. Then there’s OpenTelemetry, which has been omnipresent in the past year, especially when the project went GA at KubeCon last November and with the recent addition of profiling as an OpenTelemetry signal.
We believe that the most operationally sound observability system looks like a blend of these two leading open source technologies. Alloy is compatible with both the OpenTelemetry Collector and Prometheus Agent, which means it can be used as an alternative to either of these solutions or combined into a hybrid system of multiple collectors and agents. Alloy can also accept existing vanilla OpenTelemetry Collector and Prometheus Agent configuration files natively or convert them automatically to the Alloy configuration syntax for advanced use cases.
Alloy can also be used with Grafana Cloud or any compatible backend from other vendors. You can configure Alloy with your observability solution to fit on-prem, cloud only, or a mix of both. It’s also flexible enough to cater to different users, large and small, at different phases of their observability journey.
Alloy: More than observability signals
Alloy, however, is not just about telemetry data. We created Alloy to do much more:
- Production-ready pipelines: Alloy modules can distill an unwieldy, large config file into a single module that’s just a few lines, which enables SREs to get started immediately.
- GitOps-friendly: Alloy introduces a framework that can pull configs from anywhere, such as GitHub, Amazon S3, and HTTP endpoints.
- Built for scale: With built-in native clustering, fleets of collectors can work together for better workload distribution and high availability against pull-based workloads. You’ll be able to create horizontally scalable deployments with minimal resource and operational overhead.
- Security: Users can connect to a HashiCorp Vault server to retrieve secrets in order to avoid managing authentication credentials and tokens locally.
- Debugging utilities: Alloy allows users to easily troubleshoot issues at configuration with an embedded UI that’s viewable from the Alloy HTTP server.
Alloy + Grafana Cloud
Alloy can work in any self-managed use case, but it’s also a natural companion with Grafana Cloud.
By pairing Alloy with Grafana Cloud, users can leverage an opinionated observability solution that’s built on top of the very best open source observability projects. Alloy makes it even easier to connect all your telemetry data to Grafana Cloud to power these solutions:
- Grafana Cloud Collector UI: The brand new home for Alloy to easily manage your connected collectors and quickly check in on their health and performance.
- Application Observability: Our solution for observing and monitoring the performance of your organization’s applications. Powered by Grafana Cloud, OpenTelemetry semantic conventions, and the Prometheus data model, Application Observability allows you to detect anomalies, identify root causes, and fix problems faster for a reduced MTTR. We also make it easier to instrument your applications with Alloy by directly embedding Grafana Beyla, our open source eBPF-based auto-instrumentation tool that is compatible with OpenTelemetry and Prometheus.
- Kubernetes Monitoring: Our premiere solution for infrastructure monitoring that makes it easier to analyze your Kubernetes environment from the cluster level down to an individual pod. You can get started with a simple Helm chart that contains ready-to-go instructions for Alloy.
- Out-of-the-box integrations: Grafana Cloud maintains a catalog of more than 100 integrations that enable monitoring instantly with prebuilt configs, dashboards, and alerts. Alloy users can mix and match between instrumenting for custom telemetry across both OpenTelemetry and Prometheus signals, combined with integrations for popular services such as Linux, AWS, Docker, and more. Expect to see full Alloy compatibility for these Grafana Cloud solutions to roll out over the coming weeks.
What’s next for Grafana Alloy
We’re not done yet! Following this v1.0 launch announcement, we’re already planning to introduce the next wave of exciting features.
Improved integration with Helm and Ansible
Grafana Alloy introduces a new configuration syntax that makes it easier to compose pipelines. We’re actively exploring optional support for YAML-based syntax to enable Alloy to better integrate with existing tooling like Helm and Ansible. Expect to see some news here in the coming weeks!
Fleet management
For Grafana Cloud users, expect to see a fleet management solution that simplifies installation and configuration management for all your collectors. The managed solution can be called via API or you can interact with it directly in a GUI. Users will then be able to create, update, and manage their collector configurations centrally without having to manually edit files by individual instance.
Health meta-monitoring
Current users of Grafana Cloud may already be familiar with the Grafana Agent health integration, which makes it easy to monitor the health and performance of your many connected agents. We’re bringing the same out-of-the-box experience to Alloy within Grafana Cloud and elevating the experience beyond just dashboards. Expect to see a curated experience that helps you better understand the health and state of your collectors.
Get started with Grafana Alloy today
We have a series of migration guides and tools to help you easily get started with Grafana Alloy.
As always, community feedback is welcome and appreciated! Please find us in #alloy channel in the Grafana Labs Community Slack or check out Grafana Alloy in GitHub.
