Documentation Grafana Cloud Monitor infrastructure Grafana integrations Integrations reference Varnish Cache

Varnish Cache integration for Grafana Cloud

Varnish Cache is an open source web application accelerator, also called a caching HTTP reverse proxy. This integration for Grafana Cloud allows users to collect metrics and logs for their Varnish system.

Varnish Cache integration for Grafana cloud provides a streamlined way for users to easily monitor their Varnish system with little overhead while getting the crucial information they need.

This integration supports Varnish Cache 6.0 LTS.

This integration includes 6 useful alerts and 1 pre-built dashboard to help monitor and visualize Varnish Cache metrics and logs.

Before you begin

In order for the integration to properly work, you must set up the Varnish Cache exporter for Prometheus to export metrics from Varnish.

Set up Varnish Cache exporter

Install the version of the exporter that corresponds to your operating system. The latest releases and installs can be found on exporter installation. To start the exporter first unzip the file that you downloaded above and traverse to the directory where the executable is. The executable will be named similar to prometheus_varnish_exporter. An example of the exporter being run is shown below:

Bash Copy

sudo ./prometheus_varnish_exporter

The above will run a

Bash Copy

varnishstat -j

behind the scenes and open a port 9131 that can be accessed through localhost:9131. The exported Prometheus metrics can be seen at localhost:9131/metrics.

The exporter exposes port 9131 by default but you can change this using the -web.listen-address flag when running the exporter. The use of this flag would look like this:

Bash Copy

sudo ./prometheus_varnish_exporter -web.listen-address localhost:9595

If you have multiple Varnish instances, you must specify which you are targeting with the exporter. This can be done by running the exporter with the flag -n and the name of your varnish instance. Example shown below:

Bash Copy

sudo ./prometheus_varnish_exporter -n <instance_name>

This also means you would need one exporter for each instance of Varnish Cache, each with a unique port.

For more options and help with the exporter, you can run the below command:

Bash Copy

sudo ./prometheus_varnish_exporter -h

Once deployed, Grafana Alloy should be able to reach the Varnish Cache exporter’s exposed endpoint in order to scrape metrics.

Set up Varnish Cache logging

This integration leverages the varnishncsa utility provided by Varnish in order to produce formatted logs. Any format of log collection with varnishncsa will be picked up by the integration.

In order to setup varnishncsa, the utility must be enabled by running:

Bash Copy

sudo systemctl enable varnishncsa
sudo systemctl start varnishncsa

By default this integration provides two commands that enable frontend and backend logging capabilities. Since Varnish does not naturally record logs into a file, a log path must be specified for varnishncsa to write to. For darwin and linux systems respectively, the paths need to be /opt/varnish/log and /var/log/varnish. Note: replace <instance_name> and <log_path> with the respective names of the instance and log path. If there is only one instance, then the -n <instance_name> flag can be removed. The below commands must be re-run for each instance that log collection is required for.

Bash Copy

sudo varnishncsa -n <instance_name> -F '{"Timestamp": "%t", "Varnish-Side": "%{Varnish:side}x", "Age": %{age}o, "Handling": "%{Varnish:handling}x", "Request": "%r", "Status": "%s", "Response-Reason": "%{VSL:RespReason}x", "Fetch-Error": "%{VSL:FetchError}x", "X-Forwarded-For": "%{x-forwarded-for}i", "Remote-User": "%u", "Bytes": "%b", "Time-To-Serve": %D, "User-Agent": "%{User-agent}i", "Referer": "%{Referer}i", "X-Varnish": "%{x-varnish}o", "X-Magento-Tags": "%{x-magento-tags}o"}}' -D -w <log_path>/varnishncsa-frontend.log

sudo varnishncsa -n <instance_name> -b -F '{"Timestamp": "%t", "Varnish-Side": "%{Varnish:side}x", "Handling": "%{Varnish:handling}x", "Request": "%r", "Status": "%s", "Response-Reason": "%{VSL:RespReason}x", "Fetch-Error": "%{VSL:FetchError}x", "Bytes": "%b", "Time-To-Serve": %D}' -D -w <log_path>/varnishncsa-backend.log

For multiple instances, configure the files where logs are written to follow the following naming convention: varnishncsa-frontend-<instance_name>.log varnishncsa-backend-<instance_name>.log

Install Varnish Cache integration for Grafana Cloud

1. In your Grafana Cloud stack, click Connections in the left-hand menu.
2. Find Varnish Cache and click its tile to open the integration.
3. Review the prerequisites in the Configuration Details tab and set up Grafana Alloy to send Varnish Cache metrics and logs to your Grafana Cloud instance.
4. Click Install to add this integration’s pre-built dashboard and alerts to your Grafana Cloud instance, and you can start monitoring your Varnish Cache setup.

Configuration snippets for Grafana Alloy

Advanced mode

The following snippets provide examples to guide you through the configuration process.

To instruct Grafana Alloy to scrape your Varnish Cache instances, manually copy and append the snippets to your alloy configuration file, then follow subsequent instructions.

Advanced metrics snippets

alloy Copy

discovery.relabel "metrics_integrations_integrations_varnish_cache" {
	targets = [{
		__address__ = "<exporter_host>:<exporter_port>",
	}]

	rule {
		target_label = "instance"
		replacement  = constants.hostname
	}
}

prometheus.scrape "metrics_integrations_integrations_varnish_cache" {
	targets    = discovery.relabel.metrics_integrations_integrations_varnish_cache.output
	forward_to = [prometheus.remote_write.metrics_service.receiver]
	job_name   = "integrations/varnish-cache"
}

To monitor your Varnish Cache instance, you must use a discovery.relabel component to discover your Varnish Cache Prometheus endpoint and apply appropriate labels, followed by a prometheus.scrape component to scrape it.

Configure the following properties within each discovery.relabel component:

• __address__: The address to your Varnish Cache Prometheus metrics endpoint.
• instance label: constants.hostname sets the instance label to your Grafana Alloy server hostname. If that is not suitable, change it to a value uniquely identifies this Varnish Cache instance. Make sure this label value is the same for all telemetry data collected for this instance.

If you have multiple Varnish Cache servers to scrape, configure one discovery.relabel for each and scrape them by including each under targets within the prometheus.scrape component.

Advanced logs snippets

darwin

alloy Copy

local.file_match "logs_integrations_integrations_varnish_cache" {
	path_targets = [{
		__address__ = "localhost",
		__path__    = "<log_path>/varnishncsa*.log",
		instance    = constants.hostname,
		job         = "integrations/varnish-cache",
	}]
}

loki.source.file "logs_integrations_integrations_varnish_cache" {
	targets    = local.file_match.logs_integrations_integrations_varnish_cache.targets
	forward_to = [loki.write.grafana_cloud_loki.receiver]
}

To monitor your Varnish Cache instance logs, you will use a combination of the following components:

• local.file_match defines where to find the log file to be scraped. Change the following properties according to your environment:

  • __address__: The Varnish Cache instance address
  • __path__: The path to the log file.
  • instance label: constants.hostname sets the instance label to your Grafana Alloy server hostname. If that is not suitable, change it to a value uniquely identifies this Varnish Cache instance. Make sure this label value is the same for all telemetry data collected for this instance.

• loki.source.file sends logs to Loki.

Kubernetes log collection

These instructions assume the use of the official Varnish helm chart.

In the sample application, we generated YAML from the official helm chart and customized it in the following ways:

1. Add an initContainer that mounts the volume for logs and creates the log files to be written to.
2. Add two varnishncsa sidecars, one for frontend logs, the other for backend logs.
3. Add one Alloy sidecar with appropriate volumeMounts to the log files and the custom Alloy configuration

The Varnish cache container, the initContainer, and the varnishncsa sidecars must be run as the varnishlog user. In the following YAML examples, you’ll notice runAsUser is set to the varnishlog user ID, which may be different in your environment.

Log file initContainer

This example is for one instance, where only one log file exists per log type.

Here’s an example snippet for creating an initContainer that mounts the log file location and creates the necessary log files:

YAML Copy

initContainers:
- name: init-script
  image: alpine:latest
  securityContext:
    runAsNonRoot: true
    runAsUser: 103
  command: ['sh', '-c', 'touch /var/log/varnish/varnishncsa-frontend.log /var/log/varnish/varnishncsa-backend.log && chmod 666 /var/log/varnish/*.log']
  volumeMounts:
  - name: varnish-log
    mountPath: /var/log/varnish

Varnish log sidecars

The varnishncsa sidecars for logging need to be deployed with the appropriate arguments and volume mounts. Refer to the this example deployment for details on the varnishncsa arguments.

Here is an example snippet for the frontend logs sidecar:

YAML Copy

- name: varnish-cache-ncsa-frontend
  securityContext:
    runAsNonRoot: true
    runAsUser: 103
  image: {{ .Values.varnishImage }}
  imagePullPolicy: IfNotPresent
  command: ["/bin/sh", "-c"]
  args: ["varnishncsa -n /var/lib/varnish -F '{\"Timestamp\": \"%t\", \"Varnish-Side\": \"%{Varnish:side}x\", \"Age\": %{age}o, \"Handling\": \"%{Varnish:handling}x\", \"Request\": \"%r\", \"Status\": \"%s\", \"Response-Reason\": \"%{VSL:RespReason}x\", \"Fetch-Error\": \"%{VSL:FetchError}x\", \"X-Forwarded-For\": \"%{x-forwarded-for}i\", \"Remote-User\": \"%u\", \"Bytes\": \"%b\", \"Time-To-Serve\": %D, \"User-Agent\": \"%{User-agent}i\", \"Referer\": \"%{Referer}i\", \"X-Varnish\": \"%{x-varnish}o\", \"X-Magento-Tags\": \"%{x-magento-tags}o\"}' -w /var/log/varnish/varnishncsa-frontend.log"]
  volumeMounts:
  - name: custom-varnish-vsm
    mountPath: /var/lib/varnish
    readOnly: true
  - name: varnish-secret
    mountPath: /etc/varnish/secret
    subPath: secret
  - name: varnish-log
    mountPath: /var/log/varnish

Alloy sidecar

The Alloy sidecar is configured to collect logs from each file. Here’s an example snippet:

YAML Copy

apiVersion: v1
kind: ConfigMap
metadata:
  name: alloy-varnish-log-config
  labels:
    alloy-config: varnish-logs
data:
  config.alloy: |
    logging {
      level  = "info"
      format = "logfmt"
    }

    loki.write "logs_integrations" {
      endpoint {
        url = "http://loki.default.svc.cluster.local:3100/loki/api/v1/push"

      }
      external_labels = {"cluster" = "my-cluster"}
    }

    local.file_match "logs_integrations_integrations_varnish_cache" {
      path_targets = [{
        __address__ = "localhost",
        __path__    = "/var/log/varnish/varnishncsa-*.log",
        instance    = "custom-varnish-cache.sample-apps.svc:9131",
        job         = "integrations/varnish-cache",
      }]
    }

    loki.source.file "logs_integrations_integrations_apache_airflow" {
      targets    = local.file_match.logs_integrations_integrations_varnish_cache.targets
      forward_to = [loki.write.logs_integrations.receiver]
    }

linux

alloy Copy

local.file_match "logs_integrations_integrations_varnish_cache" {
	path_targets = [{
		__address__ = "localhost",
		__path__    = "<log_path>/varnishncsa*.log",
		instance    = constants.hostname,
		job         = "integrations/varnish-cache",
	}]
}

loki.source.file "logs_integrations_integrations_varnish_cache" {
	targets    = local.file_match.logs_integrations_integrations_varnish_cache.targets
	forward_to = [loki.write.grafana_cloud_loki.receiver]
}

To monitor your Varnish Cache instance logs, you will use a combination of the following components:

• local.file_match defines where to find the log file to be scraped. Change the following properties according to your environment:

  • __address__: The Varnish Cache instance address
  • __path__: The path to the log file.
  • instance label: constants.hostname sets the instance label to your Grafana Alloy server hostname. If that is not suitable, change it to a value uniquely identifies this Varnish Cache instance. Make sure this label value is the same for all telemetry data collected for this instance.

• loki.source.file sends logs to Loki.

Kubernetes log collection

These instructions assume the use of the official Varnish helm chart.

In the sample application, we generated YAML from the official helm chart and customized it in the following ways:

1. Add an initContainer that mounts the volume for logs and creates the log files to be written to.
2. Add two varnishncsa sidecars, one for frontend logs, the other for backend logs.
3. Add one Alloy sidecar with appropriate volumeMounts to the log files and the custom Alloy configuration

The Varnish cache container, the initContainer, and the varnishncsa sidecars must be run as the varnishlog user. In the following YAML examples, you’ll notice runAsUser is set to the varnishlog user ID, which may be different in your environment.

Log file initContainer

This example is for one instance, where only one log file exists per log type.

Here’s an example snippet for creating an initContainer that mounts the log file location and creates the necessary log files:

YAML Copy

initContainers:
- name: init-script
  image: alpine:latest
  securityContext:
    runAsNonRoot: true
    runAsUser: 103
  command: ['sh', '-c', 'touch /var/log/varnish/varnishncsa-frontend.log /var/log/varnish/varnishncsa-backend.log && chmod 666 /var/log/varnish/*.log']
  volumeMounts:
  - name: varnish-log
    mountPath: /var/log/varnish

Varnish log sidecars

The varnishncsa sidecars for logging need to be deployed with the appropriate arguments and volume mounts. Refer to the this example deployment for details on the varnishncsa arguments.

Here is an example snippet for the frontend logs sidecar:

YAML Copy

- name: varnish-cache-ncsa-frontend
  securityContext:
    runAsNonRoot: true
    runAsUser: 103
  image: {{ .Values.varnishImage }}
  imagePullPolicy: IfNotPresent
  command: ["/bin/sh", "-c"]
  args: ["varnishncsa -n /var/lib/varnish -F '{\"Timestamp\": \"%t\", \"Varnish-Side\": \"%{Varnish:side}x\", \"Age\": %{age}o, \"Handling\": \"%{Varnish:handling}x\", \"Request\": \"%r\", \"Status\": \"%s\", \"Response-Reason\": \"%{VSL:RespReason}x\", \"Fetch-Error\": \"%{VSL:FetchError}x\", \"X-Forwarded-For\": \"%{x-forwarded-for}i\", \"Remote-User\": \"%u\", \"Bytes\": \"%b\", \"Time-To-Serve\": %D, \"User-Agent\": \"%{User-agent}i\", \"Referer\": \"%{Referer}i\", \"X-Varnish\": \"%{x-varnish}o\", \"X-Magento-Tags\": \"%{x-magento-tags}o\"}' -w /var/log/varnish/varnishncsa-frontend.log"]
  volumeMounts:
  - name: custom-varnish-vsm
    mountPath: /var/lib/varnish
    readOnly: true
  - name: varnish-secret
    mountPath: /etc/varnish/secret
    subPath: secret
  - name: varnish-log
    mountPath: /var/log/varnish

Alloy sidecar

The Alloy sidecar is configured to collect logs from each file. Here’s an example snippet:

YAML Copy

apiVersion: v1
kind: ConfigMap
metadata:
  name: alloy-varnish-log-config
  labels:
    alloy-config: varnish-logs
data:
  config.alloy: |
    logging {
      level  = "info"
      format = "logfmt"
    }

    loki.write "logs_integrations" {
      endpoint {
        url = "http://loki.default.svc.cluster.local:3100/loki/api/v1/push"

      }
      external_labels = {"cluster" = "my-cluster"}
    }

    local.file_match "logs_integrations_integrations_varnish_cache" {
      path_targets = [{
        __address__ = "localhost",
        __path__    = "/var/log/varnish/varnishncsa-*.log",
        instance    = "custom-varnish-cache.sample-apps.svc:9131",
        job         = "integrations/varnish-cache",
      }]
    }

    loki.source.file "logs_integrations_integrations_apache_airflow" {
      targets    = local.file_match.logs_integrations_integrations_varnish_cache.targets
      forward_to = [loki.write.logs_integrations.receiver]
    }

Kubernetes instructions

Instructions for Kubernetes

Dashboards

The Varnish Cache integration installs the following dashboards in your Grafana Cloud instance to help monitor your system.

• Varnish overview

Varnish Cache overview (overview)

Varnish Cache overview (logs)

Alerts

The Varnish Cache integration includes the following useful alerts:

Metrics

The most important metrics provided by the Varnish Cache integration, which are used on the pre-built dashboard and Prometheus alerts, are as follows:

• up
• varnish_backend_beresp_bodybytes
• varnish_backend_beresp_hdrbytes
• varnish_main_backend_busy
• varnish_main_backend_conn
• varnish_main_backend_recycle
• varnish_main_backend_req
• varnish_main_backend_reuse
• varnish_main_backend_unhealthy
• varnish_main_cache_hit
• varnish_main_cache_hitpass
• varnish_main_cache_miss
• varnish_main_client_req
• varnish_main_n_expired
• varnish_main_n_lru_nuked
• varnish_main_pools
• varnish_main_s_resp_bodybytes
• varnish_main_s_resp_hdrbytes
• varnish_main_sessions
• varnish_main_sessions_total
• varnish_main_thread_queue_len
• varnish_main_threads
• varnish_main_threads_created
• varnish_main_threads_failed
• varnish_main_threads_limited
• varnish_sma_g_bytes
• varnish_sma_g_space

Changelog

md Copy

# 1.0.0 - August 2024
* Add Kubernetes as a supported platform using Grafana Alloy

# 0.0.3 - September 2023

* New Filter Metrics option for configuring the Grafana Agent, which saves on metrics cost by dropping any metric not used by this integration. Beware that anything custom built using metrics that are not on the snippet will stop working.
* New hostname relabel option, which applies the instance name you write on the text box to the Grafana Agent configuration snippets, making it easier and less error prone to configure this mandatory label.

# 0.0.2 - August 2023

* Add regex filter for logs datasource

# 0.0.1 - July 2023

* Initial release

Cost

By connecting your Varnish Cache instance to Grafana Cloud, you might incur charges. To view information on the number of active series that your Grafana Cloud account uses for metrics included in each Cloud tier, see Active series and dpm usage and Cloud tier pricing.