Hadoop HDFS FSImage
Used with Prometheus Hadoop HDFS FSImage Exporter in kubernetes
About
Initial dashboard for hadoop in kubernetes (wait what?)
This is mainly a test of hdfs + nn + zk + jn in k8s:
- k8s 1.15.x
- prometheus v2.17.2 via prometheus-operator
- grafana v6.7.3 (a04ef6cefc) with prometheus as Data Source
- Hadoop 3.1.3 in HA setup ( zookeeper cluster + 3 journalnodes + 3 namenodes, which means 1 active nn and 2 standby)
- marcelmay/hadoop-hdfs-fsimage-exporter v1.3 as sidecar in namenode pods, version 1.3 required for hadoop 3.x
- ServiceMonitor to scrape metrics - you must add ti on your own.
Known limitations
- You need to create service monitor on your own.
- You may need to adjust dashboard to match your prometheus labels
- Vast majority of metrics is not yet exposed on the graphs, but you can add them on your own
Changelog
- added variable for DS_PROMETHEUS so that json file can be directly added to grafana data directory and solves errors such as
Templating init failed Datasource named ${DS_PROMETHEUS} was not found
`
- initial release
Contact
- contact author if you have any suggestions etc
Dashboard revisions
Upload an updated version of an exported dashboard.json file from Grafana
Revision | Decscription | Created | |
---|---|---|---|
Download |
Sign up for Grafana Cloud
Get this dashboard
Data source:
Dependencies: