Grafana Cloud

ClickHouse integration for Grafana Cloud

ClickHouse is an open-source column-oriented database management system that allows generating analytical data reports in real-time. The ClickHouse integration uses the Grafana agent to collect metrics to monitor a ClickHouse cluster that can be used to easily select metrics via cluster instance. The integration also supports the ClickHouse error logs being scraped by the agent using Promtail. An accompanying dashboard is provided to visualize these metrics and logs.

This integration supports ClickHouse

Install ClickHouse integration for Grafana Cloud

  1. In your Grafana Cloud instance, click on Connections from the Home menu located near the top left-hand side of the page.
  2. Navigate to the ClickHouse tile and click on it to review the prerequisites.
  3. When you are ready, follow the steps on the Configuration Details page to setup Grafana Agent and start sending ClickHouse metrics to your Grafana Cloud instance.

Pre-install configuration for the ClickHouse integration

ClickHouse supports multi-file configurations using either a /etc/clickhouse-server/config.xml or /etc/clickhouse-server/config.yaml. This example shows config.xml snippets required to collect metrics and logs.

Prometheus metrics are scraped using the ClickHouse Exporter that must be configured in the config.xml. Some ClickHouse installations have the prometheus exporter commented out, else include this snippet:


For more information please check out Prometheus.

ClickHouse error logs are scraped using a Promtail agent that targets the default /var/log/clickhouse-server/clickhouse-server.err.log path. Logs are enabled by including the target path inside the <errorlog> within the ClickHouse config.xml. Some ClickHouse installations have the error log by default, which would be located at the top of the config.xml file.


For more information please check ClickHouse logger settings.

After editing the config.xml, restart the ClickHouse server:

sudo service clickhouse-server restart

Post-install configuration for the ClickHouse integration

This integration supports metrics and logs from a ClickHouse instance. The Grafana Agent config is stored in /etc/grafana-agent.yaml. In order to see these signals correlated on the same dashboard, job and instance labels must match for metrics scrape config and the logs scrape config in the Agent configuration file (replace with an identifier for your instance):

  wal_directory: /tmp/wal
    - name: integrations
        - job_name: integrations/clickhouse
            - replacement: "<your-instance-name>"
              target_label: instance
            - targets: ['localhost:9363']
        - url: http://cortex:9009/api/prom/push

  - name: integrations
      - url: http://loki:3100/loki/api/v1/push
      filename: /var/lib/grafana-agent/logs/positions.yaml
    - job_name: integrations/clickhouse
      - targets:
        - localhost
          instance: "<your-instance-name>"
          job: integrations/clickhouse
          __path__: /var/log/clickhouse-server/clickhouse-server.err.log
      - multiline:
          firstline: '^([\d]{4}).([\d]{1,2}).([\d]{1,2}) (([\d]+):([\d]+):([\d]+).([\d]+))'
      - regex:
          expression: '(?P<timestamp>(([\d]{4}).([\d]{1,2}).([\d]{1,2}) (([\d]+):([\d]+):([\d]+).([\d]+))) (?P<message>(?s:.*))$)'

Additionally, to gather logs from a Linux host using the Grafana Agent, add a Grafana Agent user as an owner to collect from the log location as mentioned here. For example, if you are running on Debian:

sudo usermod -a -G clickhouse grafana-agent


This integration includes the following dashboards.

  • ClickHouse latency
  • ClickHouse overview
  • ClickHouse replica

ClickHouse Overview Dashboard 1


ClickHouse Overview Dashboard 2


ClickHouse Latency Dashboard


ClickHouse Replica Dashboard



Below is a list of the metrics used on the dashboards and in alerts of the integration and are automatically written to your Grafana Cloud instance.

  • ClickHouseAsyncMetrics_ReplicasMaxQueueSize
  • ClickHouseMetrics_HTTPConnection
  • ClickHouseMetrics_InterserverConnection
  • ClickHouseMetrics_MemoryTracking
  • ClickHouseMetrics_MySQLConnection
  • ClickHouseMetrics_PostgreSQLConnection
  • ClickHouseMetrics_ReadonlyReplica
  • ClickHouseMetrics_TCPConnection
  • ClickHouseMetrics_ZooKeeperRequest
  • ClickHouseMetrics_ZooKeeperSession
  • ClickHouseMetrics_ZooKeeperWatch
  • ClickHouseProfileEvents_AsyncInsertQuery
  • ClickHouseProfileEvents_DiskReadElapsedMicroseconds
  • ClickHouseProfileEvents_DiskWriteElapsedMicroseconds
  • ClickHouseProfileEvents_FailedInsertQuery
  • ClickHouseProfileEvents_FailedQuery
  • ClickHouseProfileEvents_FailedSelectQuery
  • ClickHouseProfileEvents_InsertQuery
  • ClickHouseProfileEvents_NetworkReceiveBytes
  • ClickHouseProfileEvents_NetworkSendBytes
  • ClickHouseProfileEvents_NetworkSendElapsedMicroseconds
  • ClickHouseProfileEvents_Query
  • ClickHouseProfileEvents_RejectedInserts
  • ClickHouseProfileEvents_ReplicatedPartChecks
  • ClickHouseProfileEvents_ReplicatedPartFetches
  • ClickHouseProfileEvents_ReplicatedPartMerges
  • ClickHouseProfileEvents_ReplicatedPartMutations
  • ClickHouseProfileEvents_SelectQuery
  • ClickHouseProfileEvents_ZooKeeperWaitMicroseconds


The integration also comes packaged with a number of handy alerts.

Group: ClickHouseAlerts

ClickHouseReplicationQueueBackingUpWarning: ClickHouse replica max queue size backing up.
ClickHouseRejectedInsertsCritical: ClickHouse has too many rejected inserts.
ClickHouseZookeeperSessionsCritical: ClickHouse has too many Zookeeper sessions.
ClickHouseReplicasInReadOnlyCritical: ClickHouse has too many replicas in read only state.


# 0.0.1 - February 2023

* Initial release


By connecting your ClickHouse instance to Grafana Cloud you might incur charges. To view information on the number of active series that your Grafana Cloud account uses for metrics included in each Cloud tier, see Active series and dpm usage and Cloud tier pricing.