Hadoop HDFS FSImage

Dashboard

Used with Prometheus Hadoop HDFS FSImage Exporter in kubernetes
Last updated: 7 months ago

Start with Grafana Cloud and the new FREE tier. Includes 10K series Prometheus or Graphite Metrics and 50gb Loki Logs

Downloads: 272

Reviews: 0

  • dashboard-fs8.png
    dashboard-fs8.png

About

Initial dashboard for hadoop in kubernetes (wait what?)

This is mainly a test of hdfs + nn + zk + jn in k8s:

  • k8s 1.15.x
  • prometheus v2.17.2 via prometheus-operator
  • grafana v6.7.3 (a04ef6cefc) with prometheus as Data Source
  • Hadoop 3.1.3 in HA setup ( zookeeper cluster + 3 journalnodes + 3 namenodes, which means 1 active nn and 2 standby)
  • marcelmay/hadoop-hdfs-fsimage-exporter v1.3 as sidecar in namenode pods, version 1.3 required for hadoop 3.x
  • ServiceMonitor to scrape metrics - you must add ti on your own.

Known limitations

  • You need to create service monitor on your own.
  • You may need to adjust dashboard to match your prometheus labels
  • Vast majority of metrics is not yet exposed on the graphs, but you can add them on your own

Changelog

  • added variable for DS_PROMETHEUS so that json file can be directly added to grafana data directory and solves errors such as Templating init failed Datasource named ${DS_PROMETHEUS} was not found `
  • initial release

Contact

Dependencies: