DocumentationGrafana CloudMonitor infrastructureGrafana integrationsIntegrations referenceAzure Event Hubs

Azure Event Hubs integration for Grafana Cloud

Move your messages from the Azure Event Hubs to Grafana Cloud.

This integration includes 1 pre-built dashboard to help monitor and visualize Azure Event Hubs metrics and logs.

Before you begin

The Azure Event Hubs integration is based on configuring Grafana Alloy to consume events from Azure Event Hub.

This integration uses Apache Kafka endpoint provided by Azure Event Hubs.

This endpoint is available for all Azure Event Hubs pricing plans except Basic. Pricing plans are described on the Microsoft website.

Example of how to set up Azure Event Hubs and send logs from Function App

This is an example of how to start Event Hubs and route the logs from the Function App.

Event Hubs

1. Creating an Azure Event Hubs namespace with an enabled Kafka endpoint and creating a policy for listening using the Azure CLI: Log in to your Azure account using the following command:
   Copy

   az login

   Follow the instructions to complete the login process.
2. List your available subscriptions by running the following command:
   Copy

   az account list --output table

   This will show a list of your available subscriptions and their details.
3. Select the subscription that you want to use by running the following command:
   Copy

   az account set --subscription <subscription-id>

   Replace with the ID of the subscription that you want to use.
4. Create an Azure Event Hubs namespace using the following command:
   Copy

   az eventhubs namespace create --resource-group <resource-group-name> --name <namespace-name> --location <location> --sku <sku> --enable-kafka true

   Replace , , , with your own values. The parameter should be set to either “Standard” or “Premium” if you want to enable Kafka.
5. Create an Event Hub within your namespace using the following command:
   Copy

   az eventhubs eventhub create --resource-group <resource-group-name> --namespace-name <namespace-name> --name <eventhub-name>

   Replace , , and with your own values.
6. Create a shared access policy for listening to events using the following command:
   Copy

   az eventhubs namespace authorization-rule create --name <rule_name> --resource-group <resource-group-name> --namespace-name <namespace-name> --rights Listen

   Replace <rule_name>, , , and with your own values. This command creates a policy with the “Listen” permission.
7. Get the connection string value for your Event Hub using the following command:
   Copy

   az eventhubs namespace authorization-rule keys list --resource-group <resource-group-name> --namespace-name <namespace-name> --name <policy-name> --query primaryConnectionString --output tsv

   Replace , , and with your own values. This command will output the connection string for your Event Hubs Namespace, which is needed to configure the integration to receive your logs. Note: Make sure to save the connection string value in a secure location, as it contains sensitive information and can be used to access all your event hubs within a namespace.

Configure diagnostic settings for Function App

To set up diagnostic settings for your Function App to send logs to an event hub, you need to follow these steps:

1. Navigate to your Function App in the Azure portal.
2. Under “Monitoring”, click on “Diagnostic settings”.
3. Click on “+ Add diagnostic setting” to add a new diagnostic setting.
4. Give your diagnostic setting a name.
5. Under “Categories”, select the category of logs that you want to send to the event hub. For example, you may choose “Function Application Logs”.
6. Under “Destination details”, select “Stream to an event hub” as the destination and choose the appropriate options.
7. Configure any advanced settings if needed.
8. Click “Save” to save your diagnostic setting.
9. Once your diagnostic setting is saved, your Function App will start sending logs to the specified event hub.

Install Azure Event Hubs integration for Grafana Cloud

1. In your Grafana Cloud stack, click Connections in the left-hand menu.
2. Find Azure Event Hubs and click its tile to open the integration.
3. Review the prerequisites in the Configuration Details tab and set up Grafana Agent to send Azure Event Hubs metrics and logs to your Grafana Cloud instance.
4. Click Install to add this integration’s pre-built dashboard to your Grafana Cloud instance, and you can start monitoring your Azure Event Hubs setup.

Configuration snippets for Grafana Alloy

Advanced mode

The following snippets provide examples to guide you through the configuration process.

To instruct Grafana Alloy to scrape your Azure Event Hubs instances, manually copy and append the snippets to your alloy configuration file, then follow subsequent instructions.

Advanced logs snippets

darwin

alloy Copy

discovery.relabel "logs_integrations_integrations_azure_event_hubs" {
	targets = []

	rule {
		source_labels = ["__meta_kafka_topic"]
		target_label  = "topic"
	}
}

loki.source.azure_event_hubs "logs_integrations_integrations_azure_event_hubs" {
	fully_qualified_namespace = "<fully_qualified_namespace>"
	event_hubs                = ["<event_hub_name>"]

	authentication {
		mechanism         = "connection_string"
		connection_string = "<connection_string>"
	}
	use_incoming_timestamp = true
	relabel_rules          = discovery.relabel.logs_integrations_integrations_azure_event_hubs.rules
	labels                 = {
		job = "integrations/azure_event_hubs",
	}
	forward_to = [loki.write.grafana_cloud_loki.receiver]
}

To monitor your Azure Event Hubs, you will use a combination of the following components:

• discovery.relabel defines any relabeling needed before sending logs to Loki.
• loki.source.azure_event_hubs receives Azure Event Hubs messages by making use of an Apache Kafka endpoint on Event Hubs.

Find the placeholders and replace them with your values:

• <fully_qualified_namespace> - Event hub namespace. The fully_qualified_namespace argument must refer to a full HOST:PORT that points to your event hub, such as <namespace>.servicebus.windows.net:9093. <namespace> is the name of your Event Hubs Namespace, and it is specified when you create the namespace. Later it can be found in the Azure portal on the Event Hubs Namespace dashboard.
• <connection_string> - Event Hubs ConnectionString for authentication on Azure Cloud. To find or create a connection string for Event Hubs in Azure, follow these steps:
  1. Navigate to your Event Hubs in the Azure portal.
  2. Under “Settings”, click on “Shared access policies” to see existing policies.
  3. Click on the “+” button to create a new shared access policy.
  4. Give your policy a name and select the appropriate permissions (at least “Listen”).
  5. Click “Create” to create the policy.
  6. Click on your newly created policy to view its properties.
  7. Under “Shared access keys”, you can view the primary and secondary connection strings for your policy.
  8. Copy the connection string that you want to use and use it in your code or configuration.
• <event_hub_name> - Event Hub to consume, you can specify multiple.

linux

alloy Copy

discovery.relabel "logs_integrations_integrations_azure_event_hubs" {
	targets = []

	rule {
		source_labels = ["__meta_kafka_topic"]
		target_label  = "topic"
	}
}

loki.source.azure_event_hubs "logs_integrations_integrations_azure_event_hubs" {
	fully_qualified_namespace = "<fully_qualified_namespace>"
	event_hubs                = ["<event_hub_name>"]

	authentication {
		mechanism         = "connection_string"
		connection_string = "<connection_string>"
	}
	use_incoming_timestamp = true
	relabel_rules          = discovery.relabel.logs_integrations_integrations_azure_event_hubs.rules
	labels                 = {
		job = "integrations/azure_event_hubs",
	}
	forward_to = [loki.write.grafana_cloud_loki.receiver]
}

To monitor your Azure Event Hubs, you will use a combination of the following components:

• discovery.relabel defines any relabeling needed before sending logs to Loki.
• loki.source.azure_event_hubs receives Azure Event Hubs messages by making use of an Apache Kafka endpoint on Event Hubs.

Find the placeholders and replace them with your values:

• <fully_qualified_namespace> - Event hub namespace. The fully_qualified_namespace argument must refer to a full HOST:PORT that points to your event hub, such as <namespace>.servicebus.windows.net:9093. <namespace> is the name of your Event Hubs Namespace, and it is specified when you create the namespace. Later it can be found in the Azure portal on the Event Hubs Namespace dashboard.
• <connection_string> - Event Hubs ConnectionString for authentication on Azure Cloud. To find or create a connection string for Event Hubs in Azure, follow these steps:
  1. Navigate to your Event Hubs in the Azure portal.
  2. Under “Settings”, click on “Shared access policies” to see existing policies.
  3. Click on the “+” button to create a new shared access policy.
  4. Give your policy a name and select the appropriate permissions (at least “Listen”).
  5. Click “Create” to create the policy.
  6. Click on your newly created policy to view its properties.
  7. Under “Shared access keys”, you can view the primary and secondary connection strings for your policy.
  8. Copy the connection string that you want to use and use it in your code or configuration.
• <event_hub_name> - Event Hub to consume, you can specify multiple.

windows

alloy Copy

discovery.relabel "logs_integrations_integrations_azure_event_hubs" {
	targets = []

	rule {
		source_labels = ["__meta_kafka_topic"]
		target_label  = "topic"
	}
}

loki.source.azure_event_hubs "logs_integrations_integrations_azure_event_hubs" {
	fully_qualified_namespace = "<fully_qualified_namespace>"
	event_hubs                = ["<event_hub_name>"]

	authentication {
		mechanism         = "connection_string"
		connection_string = "<connection_string>"
	}
	use_incoming_timestamp = true
	relabel_rules          = discovery.relabel.logs_integrations_integrations_azure_event_hubs.rules
	labels                 = {
		job = "integrations/azure_event_hubs",
	}
	forward_to = [loki.write.grafana_cloud_loki.receiver]
}

To monitor your Azure Event Hubs, you will use a combination of the following components:

• discovery.relabel defines any relabeling needed before sending logs to Loki.
• loki.source.azure_event_hubs receives Azure Event Hubs messages by making use of an Apache Kafka endpoint on Event Hubs.

Find the placeholders and replace them with your values:

• <fully_qualified_namespace> - Event hub namespace. The fully_qualified_namespace argument must refer to a full HOST:PORT that points to your event hub, such as <namespace>.servicebus.windows.net:9093. <namespace> is the name of your Event Hubs Namespace, and it is specified when you create the namespace. Later it can be found in the Azure portal on the Event Hubs Namespace dashboard.
• <connection_string> - Event Hubs ConnectionString for authentication on Azure Cloud. To find or create a connection string for Event Hubs in Azure, follow these steps:
  1. Navigate to your Event Hubs in the Azure portal.
  2. Under “Settings”, click on “Shared access policies” to see existing policies.
  3. Click on the “+” button to create a new shared access policy.
  4. Give your policy a name and select the appropriate permissions (at least “Listen”).
  5. Click “Create” to create the policy.
  6. Click on your newly created policy to view its properties.
  7. Under “Shared access keys”, you can view the primary and secondary connection strings for your policy.
  8. Copy the connection string that you want to use and use it in your code or configuration.
• <event_hub_name> - Event Hub to consume, you can specify multiple.

Grafana Agent static configuration (deprecated)

The following section shows configuration for running Grafana Agent in static mode which is deprecated. You should use Grafana Alloy for all new deployments.

Instructions for Grafana Agent static

Dashboards

The Azure Event Hubs integration installs the following dashboards in your Grafana Cloud instance to help monitor your system.

• Azure Event Hubs Overview

Overview

Changelog

md Copy

# 0.0.2 - August 2023

* Add regex filter for logs datasource

# 0.0.1 - April 2023

* Initial Release

Cost

By connecting your Azure Event Hubs instance to Grafana Cloud, you might incur charges. To view information on the number of active series that your Grafana Cloud account uses for metrics included in each Cloud tier, see Active series and dpm usage and Cloud tier pricing.