Configure Grafana Agentintegrations_confignode_exporter_config

node_exporter_config

The node_exporter_config block configures the node_exporter integration, which is an embedded version of node_exporter and allows for collecting metrics from the UNIX system that node_exporter is running on. It provides a significant amount of collectors that are responsible for monitoring various aspects of the host system.

Note that if running the Agent in a container, you will need to bind mount folders from the host system so the integration can monitor them. You can use the example below, making sure to replace /path/to/config.yaml with a path on your host machine where an Agent configuration file is:

docker run \
  --net="host" \
  --pid="host" \
  --cap-add=SYS_TIME \
  -v "/:/host/root:ro,rslave" \
  -v "/sys:/host/sys:ro,rslave" \
  -v "/proc:/host/proc:ro,rslave" \
  -v /tmp/agent:/etc/agent \
  -v /path/to/config.yaml:/etc/agent-config/agent.yaml \
  grafana/agent:v0.19.0 \
  --config.file=/etc/agent-config/agent.yaml

Use this configuration file for testing out node_exporter support, replacing the remote_write settings with settings appropriate for you:

server:
  log_level: info
  http_listen_port: 12345

metrics:
  wal_directory: /tmp/agent
  global:
    scrape_interval: 15s
    remote_write:
    - url: https://prometheus-us-central1.grafana.net/api/prom/push
      basic_auth:
        username: user-id
        password: api-token

integrations:
  node_exporter:
    enabled: true
    rootfs_path: /host/root
    sysfs_path: /host/sys
    procfs_path: /host/proc

For running on Kubernetes, ensure to set the equivalent mounts and capabilities there as well:

apiVersion: v1
kind: Pod
metadata:
  name: agent
spec:
  containers:
  - image: grafana/agent:v0.19.0
    name: agent
    args:
    - --config.file=/etc/agent-config/agent.yaml
    securityContext:
      capabilities:
        add: ["SYS_TIME"]
      priviliged: true
      runAsUser: 0
    volumeMounts:
    - name: rootfs
      mountPath: /host/root
      readOnly: true
    - name: sysfs
      mountPath: /host/sys
      readOnly: true
    - name: procfs
      mountPath: /host/proc
      readOnly: true
  hostPID: true
  hostNetwork: true
  dnsPolicy: ClusterFirstWithHostNet
  volumes:
  - name: rootfs
    hostPath:
      path: /
  - name: sysfs
    hostPath:
      path: /sys
  - name: procfs
    hostPath:
      path: /proc

The manifest and Tanka configs provided by this repository do not have the mounts or capabilities required for running this integration.

Some collectors only work on specific operating systems, documented in the table below. Enabling a collector that is not supported by the operating system the Agent is running on is a no-op.

Name Description OS Enabled by default
arp Exposes ARP statistics from /proc/net/arp. Linux yes
bcache Exposes bcache statistics from /sys/fs/bcache. Linux yes
bonding Exposes the number of configured and active slaves of Linux bonding interfaces. Linux yes
boottime Exposes system boot time derived from the kern.boottime sysctl. Darwin, Dragonfly, FreeBSD, NetBSD, OpenBSD, Solaris yes
btrfs Exposes statistics on btrfs. Linux yes
buddyinfo Exposes statistics of memory fragments as reported by /proc/buddyinfo. Linux no
conntrack Shows conntrack statistics (does nothing if no /proc/sys/net/netfilter/ present). Linux yes
cpu Exposes CPU statistics. Darwin, Dragonfly, FreeBSD, Linux, Solaris yes
cpufreq Exposes CPU frequency statistics. Linux, Solaris yes
devstat Exposes device statistics. Dragonfly, FreeBSD no
diskstats Exposes disk I/O statistics. Darwin, Linux, OpenBSD yes
drbd Exposes Distributed Replicated Block Device statistics (to version 8.4). Linux no
edac Exposes error detection and correction statistics. Linux yes
entropy Exposes available entropy. Linux yes
exec Exposes execution statistics. Dragonfly, FreeBSD yes
filefd Exposes file descriptor statistics from /proc/sys/fs/file-nr. Linux yes
filesystem Exposes filesystem statistics, such as disk space used. Darwin, Dragonfly, FreeBSD, Linux, OpenBSD yes
hwmon Exposes hardware monitoring and sensor data from /sys/class/hwmon. Linux yes
infiniband Exposes network statistics specific to InfiniBand and Intel OmniPath configurations. Linux yes
interrupts Exposes detailed interrupts statistics. Linux, OpenBSD no
ipvs Exposes IPVS status from /proc/net/ip_vs and stats from /proc/net/ip_vs_stats. Linux yes
ksmd Exposes kernel and system statistics from /sys/kernel/mm/ksm. Linux no
loadavg Exposes load average. Darwin, Dragonfly, FreeBSD, Linux, NetBSD, OpenBSD, Solaris yes
logind Exposes session counts from logind. Linux no
mdadm Exposes statistics about devices in /proc/mdstat (does nothing if no /proc/mdstat present). Linux yes
meminfo Exposes memory statistics. Darwin, Dragonfly, FreeBSD, Linux, OpenBSD yes
meminfo_numa Exposes memory statistics from /proc/meminfo_numa. Linux no
mountstats Exposes filesystem statistics from /proc/self/mountstats. Exposes detailed NFS client statistics. Linux no
netclass Exposes network interface info from /sys/class/net. Linux yes
netdev Exposes network interface statistics such as bytes transferred. Darwin, Dragonfly, FreeBSD, Linux, OpenBSD yes
netstat Exposes network statistics from /proc/net/netstat. This is the same information as netstat -s. Linux yes
nfs Exposes NFS client statistics from /proc/net/rpc/nfs. This is the same information as nfsstat -c. Linux yes
nfsd Exposes NFS kernel server statistics from /proc/net/rpc/nfsd. This is the same information as nfsstat -s. Linux yes
ntp Exposes local NTP daemon helath to check time. any no
perf Exposes perf based metrics (Warning: Metrics are dependent on kernel configuration and settings). Linux no
powersupplyclass Collects information on power supplies. any yes
pressure Exposes pressure stall statistics from /proc/pressure/. Linux (kernel 4.20+ and/or CONFIG_PSI) yes
processes Exposes aggregate process statistics from /proc. Linux no
qdisc Exposes queuing discipline statistics. Linux no
rapl Exposes various statistics from /sys/class/powercap. Linux yes
runit Exposes service status from runit. any no
schedstat Exposes task scheduler statistics from /proc/schedstat. Linux yes
sockstat Exposes various statistics from /proc/net/sockstat. Linux yes
softnet Exposes statistics from /proc/net/softnet_stat. Linux yes
stat Exposes various statistics from /proc/stat. This includes boot time, forks and interrupts. Linux yes
supervisord Exposes service status from supervisord. any no
systemd Exposes service and system status from systemd. Linux no
tcpstat Exposes TCP connection status information from /proc/net/tcp and /proc/net/tcp6. (Warning: the current version has potential performance issues in high load situations). Linux no
textfile Collects metrics from files in a directory matching the filename pattern *.prom. The files must be using the text format defined here: https://prometheus.io/docs/instrumenting/exposition_formats/ any yes
thermal_zone Exposes thermal zone & cooling device statistics from /sys/class/thermal. Linux yes
time Exposes the current system time. any yes
timex Exposes selected adjtimex(2) system call stats. Linux yes
udp_queues Exposes UDP total lengths of the rx_queue and tx_queue from /proc/net/udp and /proc/net/udp6. Linux yes
uname Exposes system information as provided by the uname system call. Darwin, FreeBSD, Linux, OpenBSD yes
vmstat Exposes statistics from /proc/vmstat. Linux yes
wifi Exposes WiFi device and station statistics. Linux no
xfs Exposes XFS runtime statistics. Linux (kernel 4.4+) yes
zfs Exposes ZFS performance statistics. Linux, Solaris yes
  # Enables the node_exporter integration, allowing the Agent to automatically
  # collect system metrics from the host UNIX system.
  [enabled: <boolean> | default = false]

  # Automatically collect metrics from this integration. If disabled,
  # the node_exporter integration will be run but not scraped and thus not remote-written. Metrics for the
  # integration will be exposed at /integrations/node_exporter/metrics and can
  # be scraped by an external process.
  [scrape_integration: <boolean> | default = <integrations_config.scrape_integrations>]

  # How often should the metrics be collected? Defaults to
  # prometheus.global.scrape_interval.
  [scrape_interval: <duration> | default = <global_config.scrape_interval>]

  # The timtout before considering the scrape a failure. Defaults to
  # prometheus.global.scrape_timeout.
  [scrape_timeout: <duration> | default = <global_config.scrape_timeout>]

  # Allows for relabeling labels on the target.
  relabel_configs:
    [- <relabel_config> ... ]

  # Relabel metrics coming from the integration, allowing to drop series
  # from the integration that you don't care about.
  metric_relabel_configs:
    [ - <relabel_config> ... ]

  # How frequent to truncate the WAL for this integration.
  [wal_truncate_frequency: <duration> | default = "60m"]

  # Monitor the exporter itself and include those metrics in the results.
  [include_exporter_metrics: <boolean> | default = false]

  # Optionally defines the the list of enabled-by-default collectors.
  # Anything not provided in the list below will be disabled by default,
  # but requires at least one element to be treated as defined.
  #
  # This is useful if you have a very explicit set of collectors you wish
  # to run.
  set_collectors:
    - [<string>]

  # Additional collectors to enable on top of the default set of enabled
  # collectors or on top of the list provided by set_collectors.
  #
  # This is useful if you have a few collectors you wish to run that are
  # not enabled by default, but do not want to explicitly provide an entire
  # list through set_collectors.
  enable_collectors:
    - [<string>]

  # Additional collectors to disable on top of the default set of disabled
  # collectors. Takes precedence over enable_collectors.

  # Additional collectors to disable from the set of enabled collectors.
  # Takes precedence over enabled_collectors.
  #
  # This is useful if you have a few collectors you do not want to run that
  # are enabled by default, but do not want to explicitly provide an entire
  # list through set_collectors.
  disable_collectors:
    - [<string>]

  # procfs mountpoint.
  [procfs_path: <string> | default = "/proc"]

  # sysfs mountpoint.
  [sysfs_path: <string> | default = "/sys"]

  # rootfs mountpoint. If running in docker, the root filesystem of the host
  # machine should be mounted and this value should be changed to the mount
  # directory.
  [rootfs_path: <string> | default = "/"]

  # Enable the cpu_info metric for the cpu collector.
  [enable_cpu_info_metric: <boolean> | default = true]

  # Regexmp of devices to ignore for diskstats.
  [diskstats_ignored_devices: <string> | default = "^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\\d+n\\d+p)\\d+$"]

  # Regexp of mount points to ignore for filesystem collector.
  [filesystem_ignored_mount_points: <string> | default = "^/(dev|proc|sys|var/lib/docker/.+)($|/)"]

  # Regexp of filesystem types to ignore for filesystem collector.
  [filesystem_ignored_fs_types: <string> | default = "^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$"]

  # NTP server to use for ntp collector
  [ntp_server: <string> | default = "127.0.0.1"]

  # NTP protocol version
  [ntp_protocol_version: <int> | default = 4]

  # Certify that the server address is not a public ntp server.
  [ntp_server_is_local: <boolean> | default = false]

  # IP TTL to use wile sending NTP query.
  [ntp_ip_ttl: <int> | default = 1]

  # Max accumulated distance to the root.
  [ntp_max_distance: <duration> | default = "3466080us"]

  # Offset between local clock and local ntpd time to tolerate.
  [ntp_local_offset_tolerance: <duration> | default = "1ms"]

  # Regexp of net devices to ignore for netclass collector.
  [netclass_ignored_devices: <string> | default = "^$"]

  # Regexp of net devices to blacklist (mutually exclusive with whitelist)
  [netdev_device_blacklist: <string> | default = ""]

  # Regexp of net devices to whitelist (mutually exclusive with blacklist)
  [netdev_device_whitelist: <string> | default = ""]

  # Regexp of fields to return for netstat collector.
  [netstat_fields: <string> | default = "^(.*_(InErrors|InErrs)|Ip_Forwarding|Ip(6|Ext)_(InOctets|OutOctets)|Icmp6?_(InMsgs|OutMsgs)|TcpExt_(Listen.*|Syncookies.*|TCPSynRetrans)|Tcp_(ActiveOpens|InSegs|OutSegs|PassiveOpens|RetransSegs|CurrEstab)|Udp6?_(InDatagrams|OutDatagrams|NoPorts|RcvbufErrors|SndbufErrors))$"]

  # List of CPUs from which perf metrics should be collected.
  [perf_cpus: <string> | default = ""]

  # Regexp of power supplies to ignore for the powersupplyclass collector.
  [powersupply_ignored_supplies: <string> | default = "^$"]

  # Path to runit service directory.
  [runit_service_dir: <string> | default = "/etc/service"]

  # XML RPC endpoint for the supervisord collector.
  [supervisord_url: <string> | default = "http://localhost:9001/RPC2"]

  # Regexp of systemd units to whitelist. Units must both match whitelist
  # and not match blacklist to be included.
  [systemd_unit_whitelist: <string> | default = ".+"]

  # Regexp of systemd units to blacklist. Units must both match whitelist
  # and not match blacklist to be included.
  [systemd_unit_blacklist: <string> | default = ".+\\.(automount|device|mount|scope|slice)"]

  # Enables service unit tasks metrics unit_tasks_current and unit_tasks_max
  [systemd_enable_task_metrics: <boolean> | default = false]

  # Enables service unit metric service_restart_total
  [systemd_enable_restarts_metrics: <boolean> | default = false]

  # Enables service unit metric unit_start_time_seconds
  [systemd_enable_start_time_metrics: <boolean> | default = false]

  # Directory to read *.prom files from for the textfile collector.
  [textfile_directory: <string> | default = ""]

  # Regexp of fields to return for the vmstat collector.
  [vmstat_fields: <string> | default = "^(oom_kill|pgpg|pswp|pg.*fault).*"]