All dashboards

Hadoop HDFS FSImage

Used with Prometheus Hadoop HDFS FSImage Exporter in kubernetes

Hadoop HDFS FSImage screenshot 1

About

Initial dashboard for hadoop in kubernetes (wait what?)

This is mainly a test of hdfs + nn + zk + jn in k8s:

  • k8s 1.15.x
  • prometheus v2.17.2 via prometheus-operator
  • grafana v6.7.3 (a04ef6cefc) with prometheus as Data Source
  • Hadoop 3.1.3 in HA setup ( zookeeper cluster + 3 journalnodes + 3 namenodes, which means 1 active nn and 2 standby)
  • marcelmay/hadoop-hdfs-fsimage-exporter v1.3 as sidecar in namenode pods, version 1.3 required for hadoop 3.x
  • ServiceMonitor to scrape metrics - you must add ti on your own.

Known limitations

  • You need to create service monitor on your own.
  • You may need to adjust dashboard to match your prometheus labels
  • Vast majority of metrics is not yet exposed on the graphs, but you can add them on your own

Changelog

  • added variable for DS_PROMETHEUS so that json file can be directly added to grafana data directory and solves errors such as Templating init failed Datasource named ${DS_PROMETHEUS} was not found `
  • initial release

Contact

Revisions
RevisionDescriptionCreated
Apache Hadoop

Apache Hadoop

by Grafana Labs
Grafana Labs solution

Easily monitor your deployment of Apache Hadoop, the open source framework designed to process and store large amounts of data across distributed clusters of computers, with Grafana Cloud's out-of-the-box monitoring solution.

Learn more

Get this dashboard

Import the dashboard template

or

Download JSON

Datasource
Dependencies