Monitor applications

Database Observability

Set up

MySQL

Azure Flexible Server

Grafana Cloud

Set up Azure Database for MySQL

Set up Database Observability with Grafana Cloud to collect telemetry from Azure Database for MySQL Flexible Server using Grafana Alloy. You configure your Azure MySQL server and Alloy to forward telemetry to Grafana Cloud.

What you’ll achieve

In this article, you:

Configure Azure MySQL server parameters for monitoring.
Create monitoring users with required privileges.
Configure Alloy with the Database Observability components.
Forward telemetry to Grafana Cloud.

Before you begin

Review these requirements:

Azure Database for MySQL Flexible Server 8.0 or later.
Access to modify server parameters.
Grafana Alloy deployed and accessible to your Azure MySQL server.
Network connectivity between Alloy and your Azure MySQL server endpoint.

For general MySQL setup concepts, refer to Set up MySQL.

Configure server parameters

Enable Performance Schema and related instrumentation by configuring server parameters on your Azure Database for MySQL Flexible Server. These parameters require a server restart to take effect.

Required server parameters

Parameter	Value	Notes
`performance_schema`	`ON`	Requires restart
`performance_schema_consumer_events_waits_current`	`ON`	Requires restart
`performance_schema_consumer_events_waits_history`	`ON`	Requires restart
`performance_schema_consumer_global_instrumentation`	`ON`	Requires restart
`performance_schema_consumer_thread_instrumentation`	`ON`	Requires restart
`performance_schema_max_digest_length`	`4096`	Requires restart
`performance_schema_max_sql_text_length`	`4096`	Requires restart
`max_digest_length`	`4096`	Requires restart

Use the Azure portal

Open the Azure Portal and navigate to Azure Database for MySQL flexible servers.
Select your MySQL flexible server.
In the left menu under Settings, select Server parameters.
Search for and configure each parameter listed above.
Click Save to apply the changes.
Some parameters require a server restart. Navigate to Overview and click Restart if prompted.

For detailed portal instructions, refer to Configure server parameters in the Azure documentation.

Use Terraform

Using Terraform with azurerm_mysql_flexible_server_configuration:

resource "azurerm_mysql_flexible_server_configuration" "performance_schema" {
  name                = "performance_schema"
  resource_group_name = "<RESOURCE_GROUP>"
  server_name         = "<SERVER_NAME>"
  value               = "ON"
}

resource "azurerm_mysql_flexible_server_configuration" "performance_schema_consumer_events_waits_current" {
  name                = "performance_schema_consumer_events_waits_current"
  resource_group_name = "<RESOURCE_GROUP>"
  server_name         = "<SERVER_NAME>"
  value               = "ON"
}

resource "azurerm_mysql_flexible_server_configuration" "performance_schema_consumer_events_waits_history" {
  name                = "performance_schema_consumer_events_waits_history"
  resource_group_name = "<RESOURCE_GROUP>"
  server_name         = "<SERVER_NAME>"
  value               = "ON"
}

resource "azurerm_mysql_flexible_server_configuration" "performance_schema_consumer_global_instrumentation" {
  name                = "performance_schema_consumer_global_instrumentation"
  resource_group_name = "<RESOURCE_GROUP>"
  server_name         = "<SERVER_NAME>"
  value               = "ON"
}

resource "azurerm_mysql_flexible_server_configuration" "performance_schema_consumer_thread_instrumentation" {
  name                = "performance_schema_consumer_thread_instrumentation"
  resource_group_name = "<RESOURCE_GROUP>"
  server_name         = "<SERVER_NAME>"
  value               = "ON"
}

resource "azurerm_mysql_flexible_server_configuration" "performance_schema_max_digest_length" {
  name                = "performance_schema_max_digest_length"
  resource_group_name = "<RESOURCE_GROUP>"
  server_name         = "<SERVER_NAME>"
  value               = "4096"
}

resource "azurerm_mysql_flexible_server_configuration" "performance_schema_max_sql_text_length" {
  name                = "performance_schema_max_sql_text_length"
  resource_group_name = "<RESOURCE_GROUP>"
  server_name         = "<SERVER_NAME>"
  value               = "4096"
}

resource "azurerm_mysql_flexible_server_configuration" "max_digest_length" {
  name                = "max_digest_length"
  resource_group_name = "<RESOURCE_GROUP>"
  server_name         = "<SERVER_NAME>"
  value               = "4096"
}

Replace the placeholders:

RESOURCE_GROUP: Azure resource group name.
SERVER_NAME: Azure MySQL Flexible Server name.

Alternatively, configure parameters using the Azure CLI:

az mysql flexible-server parameter set \
  --resource-group <RESOURCE_GROUP> \
  --server-name <SERVER_NAME> \
  --name performance_schema \
  --value ON

az mysql flexible-server parameter set \
  --resource-group <RESOURCE_GROUP> \
  --server-name <SERVER_NAME> \
  --name performance_schema_consumer_events_waits_current \
  --value ON

az mysql flexible-server parameter set \
  --resource-group <RESOURCE_GROUP> \
  --server-name <SERVER_NAME> \
  --name performance_schema_consumer_events_waits_history \
  --value ON

az mysql flexible-server parameter set \
  --resource-group <RESOURCE_GROUP> \
  --server-name <SERVER_NAME> \
  --name performance_schema_consumer_global_instrumentation \
  --value ON

az mysql flexible-server parameter set \
  --resource-group <RESOURCE_GROUP> \
  --server-name <SERVER_NAME> \
  --name performance_schema_consumer_thread_instrumentation \
  --value ON

az mysql flexible-server parameter set \
  --resource-group <RESOURCE_GROUP> \
  --server-name <SERVER_NAME> \
  --name performance_schema_max_digest_length \
  --value 4096

az mysql flexible-server parameter set \
  --resource-group <RESOURCE_GROUP> \
  --server-name <SERVER_NAME> \
  --name performance_schema_max_sql_text_length \
  --value 4096

az mysql flexible-server parameter set \
  --resource-group <RESOURCE_GROUP> \
  --server-name <SERVER_NAME> \
  --name max_digest_length \
  --value 4096

Note
Some parameters require a server restart. Restart the server after applying all parameter changes.

Create a monitoring user and grant required privileges

Connect to your Azure MySQL Flexible Server and create the monitoring user:

Disable tracking of monitoring user queries

Grant object privileges for detailed data

Grant privileges to auto-enable consumers

Verify user privileges

Verify server parameter settings

Run and configure Alloy

Run Alloy and add the Database Observability configuration for your Azure MySQL server.

Run the latest Alloy version

Add the Azure MySQL configuration blocks

Add these blocks to Alloy for Azure Database for MySQL. Replace <DB_NAME>. Create a local.file with the Data Source Name string, for example, <DB_USER>:<DB_PASSWORD>@tcp(<SERVER_FQDN>:<DB_PORT>)/:

local.file "mysql_secret_<DB_NAME>" {
  filename  = "/var/lib/alloy/mysql_secret_<DB_NAME>"
  is_secret = true
}

prometheus.exporter.mysql "mysql_<DB_NAME>" {
  data_source_name  = local.file.mysql_secret_<DB_NAME>.content
  enable_collectors = ["perf_schema.eventsstatements"]
  perf_schema.eventsstatements {
    text_limit = 0
  }
}

database_observability.mysql "mysql_<DB_NAME>" {
  data_source_name  = local.file.mysql_secret_<DB_NAME>.content
  forward_to        = [loki.relabel.database_observability_mysql_<DB_NAME>.receiver]
  targets           = prometheus.exporter.mysql.mysql_<DB_NAME>.targets
  exclude_schemas = ["azure_sys", "azure_maintenance"]

  cloud_provider {
    azure {
      resource_group  = "<AZURE_RESOURCE_GROUP>"
      subscription_id = "<AZURE_SUBSCRIPTION_ID>"
      server_name     = "<AZURE_SERVER_NAME>"
    }
  }
}

loki.relabel "database_observability_mysql_<DB_NAME>" {
  forward_to = [loki.write.logs_service.receiver]

  rule {
    target_label = "instance"
    replacement  = "<INSTANCE_LABEL>"
  }
}

discovery.relabel "database_observability_mysql_<DB_NAME>" {
  targets = database_observability.mysql.mysql_<DB_NAME>.targets

  rule {
    target_label = "job"
    replacement  = "integrations/db-o11y"
  }

  rule {
    target_label = "instance"
    replacement  = "<INSTANCE_LABEL>"
  }
}

prometheus.scrape "database_observability_mysql_<DB_NAME>" {
  targets    = discovery.relabel.database_observability_mysql_<DB_NAME>.output
  forward_to = [prometheus.remote_write.metrics_service.receiver]
}

Replace the placeholders:

DB_NAME: Database name Alloy uses in component identifiers (appears in component names and secret filenames).
AZURE_RESOURCE_GROUP: Azure Resource Group for your MySQL Flexible Server.
AZURE_SUBSCRIPTION_ID: Azure Subscription ID for your MySQL Flexible Server.
AZURE_SERVER_NAME: Azure Server Name for your MySQL Flexible Server (optional).
INSTANCE_LABEL: Value that sets the instance label on logs and metrics (optional).
Secret file content DSN example: DB_USER:DB_PASSWORD@tcp(SERVER_FQDN:DB_PORT)/.
- DB_USER: Database user Alloy uses to connect (for example, db-o11y).
- DB_PASSWORD: Password for the database user.
- SERVER_FQDN: Azure MySQL server fully qualified domain name (for example, <SERVER_NAME>.mysql.database.azure.com).
- DB_PORT: Database port number (default: 3306).

Add Prometheus and Loki write configuration

Run and configure Alloy with the Grafana Kubernetes Monitoring Helm chart

Extend your values.yaml when you use the k8s-monitoring Helm chart and set databaseObservability.enabled to true within the MySQL integration.

integrations:
  collector: alloy-singleton
  mysql:
    instances:
      - name: <INSTANCE_NAME>
        jobLabel: integrations/db-o11y
        exporter:
          enabled: true
          collectors:
            perfSchemaEventsStatements:
              enabled: true
          dataSource:
            host: <SERVER_FQDN>
            auth:
              usernameKey: username
              passwordKey: password
        databaseObservability:
          enabled: true
          allowUpdatePerformanceSchemaSettings: true
          extraConfig: |
            exclude_schemas = ["azure_sys", "azure_maintenance"]
            cloud_provider {
              azure {
                resource_group  = "<AZURE_RESOURCE_GROUP>"
                subscription_id = "<AZURE_SUBSCRIPTION_ID>"
                server_name     = "<AZURE_SERVER_NAME>"
              }
            }
        secret:
          create: false
          name: <SECRET_NAME>
          namespace: <NAMESPACE>
        logs:
          enabled: true
          labelSelectors:
            app.kubernetes.io/instance: <INSTANCE_NAME>

Replace the placeholders:

INSTANCE_NAME: Name for this database instance in Kubernetes.
SERVER_FQDN: Azure MySQL server fully qualified domain name.
AZURE_RESOURCE_ID: Azure resource ID for your MySQL Flexible Server.
SECRET_NAME: Name of the Kubernetes secret containing database credentials.
NAMESPACE: Kubernetes namespace where the secret exists.

To see the full set of values, check out the k8s-monitoring Helm chart documentation or the example configuration.

Optional: Configure Azure Key Vault and Kubernetes

If you use Azure Key Vault with External Secrets Operator to manage database credentials, configure them as follows.

Secret naming convention

Store monitoring credentials in Azure Key Vault with a name following this convention:

mysql-<SERVER_NAME>-monitoring

MySQL secret format

Store the secret as JSON with the following format:

{
  "username": "db-o11y",
  "password": "<DB_O11Y_PASSWORD>",
  "host": "<SERVER_FQDN>",
  "port": 3306
}

Replace the placeholders:

DB_O11Y_PASSWORD: Password for the db-o11y MySQL user.
SERVER_FQDN: Azure MySQL server fully qualified domain name.

Create the secret via Azure CLI

az keyvault secret set \
  --vault-name <KEY_VAULT_NAME> \
  --name "mysql-<SERVER_NAME>-monitoring" \
  --value '{"username":"db-o11y","password":"<DB_O11Y_PASSWORD>","host":"<SERVER_FQDN>","port":3306}'

Kubernetes External Secrets configuration

Use the External Secrets Operator to sync the Azure secret into Kubernetes:

---
apiVersion: external-secrets.io/v1beta1
kind: SecretStore
metadata:
  name: <SERVER_NAME>-db-monitoring-secretstore
spec:
  provider:
    azurekv:
      tenantId: <AZURE_TENANT_ID>
      vaultUrl: https://<KEY_VAULT_NAME>.vault.azure.net
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: <SERVER_NAME>-db-monitoring-secret
spec:
  refreshInterval: 1h
  secretStoreRef:
    kind: SecretStore
    name: <SERVER_NAME>-db-monitoring-secretstore
  dataFrom:
    - extract:
        key: mysql-<SERVER_NAME>-monitoring

Replace the placeholders: