Apache Spark - Performance Metrics


Monitors spark metrics using Prometheus in Kubernetes
Last updated: 4 years ago

Start with Grafana Cloud and the new FREE tier. Includes 10K series Prometheus or Graphite Metrics and 50gb Loki Logs

Downloads: 2983

Reviews: 0

  • Capture.PNG

Spark Metrics Dashboard :- Having Spark Driver / Executor Memory Consumption and other related metrics in Kubernetes using JMX Exporter and Prometheus Service Discovery Including Network I/O and Disk Read/Write Metrics for Spark Driver , Executors and Shuffle Service

Add Labels to Spark Driver and Executor accordingly for Prometheus to fetch metrics through service discovery

Get this dashboard: