grafana_celery.gif
The metrics were designed as following:
| Type | Name | Worker | Task | Results |
|---|---|---|---|---|
| Gauge | workers_state | √ | ||
| Counter | workers_processed | √ | ||
| Gauge | workers_active | √ | ||
| Counter | tasks_counter | √ | √ | √ |
| Summary | tasks_runtime | √ | √ | |
| Info | tasks_info | √ | √ | √ |
The main idea is all about Prometheus with its pushgateway and its python client.
You could find more information in below.
Main project provides simple python scripts for crawling website with celery, the monitor part is defined in monitor.py;
Others like pushgateway and python client should be appreciated.
