Visualization and monitoring solutions
Visualization and monitoring solutions
/
Monitor Apache Airflow
Monitor Apache Airflow easily with Grafana
Easily monitor your deployment of Apache Airflow, an open source platform for programmatically authoring, scheduling, and monitoring workflows, with Grafana Cloud’s out-of-the-box monitoring solution. The Grafana Cloud forever-free tier includes 3 users and up to 10k metrics series to support your monitoring needs.
Apache Airflow overview (general)
Apache Airflow overview (scheduler)
Key metrics included
airflow_dag_processing_import_errors
airflow_dag_task_duration_count
airflow_dag_task_duration_sum
airflow_dagrun_duration_failed_count
airflow_dagrun_duration_failed_sum
airflow_dagrun_duration_success_count
airflow_dagrun_duration_success_sum
airflow_dagrun_schedule_delay_count
airflow_dagrun_schedule_delay_sum
airflow_executor_open_slots
airflow_executor_queued_tasks
airflow_executor_running_tasks
airflow_pool_open_slots
airflow_pool_queued_slots
airflow_pool_running_slots
airflow_pool_starving_tasks
airflow_scheduler_tasks_executable
airflow_scheduler_tasks_starving
airflow_sla_missed
airflow_task_finish_total
airflow_task_start_total
airflow_ti_failures
up
Key alerting rules included
ApacheAirflowStarvingPoolTasks (Critical)
ApacheAirflowDAGScheduleDelayWarningLevel (Warning)
ApacheAirflowDAGScheduleDelayCriticalLevel (Critical)
ApacheAirflowDAGFailures (Critical)