Menu
Grafana Cloud

Application Observability with Grafana Alloy

Grafana Alloy (formerly Grafana Agent) is a vendor-neutral distribution of the OpenTelemetry (OTel) Collector and the recommended way to send OpenTelemetry data to Grafana Cloud. Grafana Alloy will act as an outbound gateway providing reliability and scalability.

Prerequisites

To set up Grafana Alloy as a data collector and send data to Grafana Cloud:

We require that the Alloy is run on every host for Application Observability to function optimally. This also allows us to have a seamless correlation between Application and Infrastructure observability.

Configuration

An alloy-config.river configuration file is needed for the Grafana Alloy to run successfully in flow mode. It is recommended to use the OpenTelemetry (OTLP) integration to generate a configuration file.

Navigate to the OpenTelemetry Application page and click the Add service button. Choose the OpenTelemetry (OTLP) integration from the list.

OpenTelemetry Integration

The OpenTelemetry (OTLP) integration will generate a Grafana Alloy configuration to use with Grafana Application Observability:

river
otelcol.receiver.otlp "default" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.receiver.otlp/

	// configures the default grpc endpoint "0.0.0.0:4317"
	grpc { }
	// configures the default http/protobuf endpoint "0.0.0.0:4318"
	http { }

	output {
		metrics = [otelcol.processor.resourcedetection.default.input]
		logs    = [otelcol.processor.resourcedetection.default.input]
		traces  = [otelcol.processor.resourcedetection.default.input]
	}
}

otelcol.processor.resourcedetection "default" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.processor.resourcedetection/
	detectors = ["env", "system"] // add "gcp", "ec2", "ecs", "elastic_beanstalk", "eks", "lambda", "azure", "aks", "consul", "heroku"  if you want to use cloud resource detection

	system {
		hostname_sources = ["os"]
	}

	output {
		metrics = [otelcol.processor.transform.add_resource_attributes_as_metric_attributes.input]
		logs    = [otelcol.processor.batch.default.input]
		traces  = [
			otelcol.processor.batch.default.input,
			otelcol.connector.host_info.default.input,
		]
	}
}

otelcol.connector.host_info "default" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.connector.host_info/
	host_identifiers = ["host.name"]

	output {
		metrics = [otelcol.processor.batch.default.input]
	}
}

otelcol.processor.transform "add_resource_attributes_as_metric_attributes" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.processor.transform/
	error_mode = "ignore"

	metric_statements {
		context    = "datapoint"
		statements = [
			"set(attributes[\"deployment.environment\"], resource.attributes[\"deployment.environment\"])",
			"set(attributes[\"service.version\"], resource.attributes[\"service.version\"])",
		]
	}

	output {
		metrics = [otelcol.processor.batch.default.input]
	}
}

otelcol.processor.batch "default" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.processor.batch/
	output {
		metrics = [otelcol.exporter.prometheus.grafana_cloud_prometheus.input]
		logs    = [otelcol.exporter.loki.grafana_cloud_loki.input]
		traces  = [otelcol.exporter.otlp.grafana_cloud_tempo.input]
	}
}

otelcol.exporter.loki "grafana_cloud_loki" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.exporter.loki/
	forward_to = [loki.write.grafana_cloud_loki.receiver]
}

otelcol.exporter.prometheus "grafana_cloud_prometheus" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.exporter.prometheus/
	add_metric_suffixes = false
	forward_to          = [prometheus.remote_write.grafana_cloud_prometheus.receiver]
}

prometheus.remote_write "grafana_cloud_prometheus" {
	// https://grafana.com/docs/alloy/latest/reference/components/prometheus.remote_write/
	endpoint {
		url = env("GRAFANA_CLOUD_PROMETHEUS_URL")

		basic_auth {
			username = env("GRAFANA_CLOUD_PROMETHEUS_USERNAME")
			password = env("GRAFANA_CLOUD_API_KEY")
		}
	}
}

loki.write "grafana_cloud_loki" {
	// https://grafana.com/docs/alloy/latest/reference/components/loki.write/
	endpoint {
		url = env("GRAFANA_CLOUD_LOKI_URL")

		basic_auth {
			username = env("GRAFANA_CLOUD_LOKI_USERNAME")
			password = env("GRAFANA_CLOUD_API_KEY")
		}
	}
}

otelcol.exporter.otlp "grafana_cloud_tempo" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.exporter.otlp/
	client {
		endpoint = env("GRAFANA_CLOUD_TEMPO_ENDPOINT")
		auth     = otelcol.auth.basic.grafana_cloud_tempo.handler
	}
}

otelcol.auth.basic "grafana_cloud_tempo" {
	// https://grafana.com/docs/alloy/latest/reference/components/otelcol.auth.basic/
	username = env("GRAFANA_CLOUD_TEMPO_USERNAME")
	password = env("GRAFANA_CLOUD_API_KEY")
}

The configuration file requires several environmental variable to be set:

Environment VariableDescriptionExample
GRAFANA_CLOUD_API_KEYAPI key generated aboveeyJvSomeLongStringJ9fQ==
GRAFANA_CLOUD_PROMETHEUS_URLRemote Write Endpoint from Grafana Cloud > Prometheus > Detailshttps://prometheus-prod-***.grafana.net/api/prom/push
GRAFANA_CLOUD_PROMETHEUS_USERNAMEUsername/Instance ID from the Grafana Cloud > Prometheus > Details11111
GRAFANA_CLOUD_LOKI_URLurl (without the username and password) from Grafana Cloud > Loki > Details > From a standalone hosthttps://logs-prod-***.grafana.net/loki/api/v1/push
GRAFANA_CLOUD_LOKI_USERNAMEUser from Grafana Cloud > Loki > Details11112
GRAFANA_CLOUD_TEMPO_ENDPOINTendpoint from the Grafana Cloud > Tempo > Details > Sending Data to Tempotempo-***.grafana.net:443
GRAFANA_CLOUD_TEMPO_USERNAMEUser from Grafana Cloud > Tempo > Details11113

Run Grafana Alloy

Create the alloy-config.river file, set the necessary environment variables, and start Grafana Alloy.

Language Specific Guides

Language specific guides that show how to instrument your application can be found in the production instrumentation guides.