Lean, mean, monitoring machine: Grafana at every scale
Thursday, Jun 10, , UTC
Dapper Labs has used Grafana from the get-go, and managed to survive scaling our observability platform from zero to hero. From a self-hosted Grafana dashboard supporting 3 devs on a gutless Prometheus box, we’ve grown to 3 million metrics a minute streaming into Cortex, filtering insights to a data-hungry org. Dapper now covers the three pillars of observability — with the addition of Loki for logs and Tempo for tracing — all funnelled through Grafana Cloud. Considering how radically our products and monitoring needs have changed, it is perhaps surprising how little has changed in our observability stack. As the Grafana Stack expanded, we grew into it, but our core ethos remains constant. Some of this can be attributed to making the right choices up front, and some to the inherent flexibility your monitoring tools give you when done right. What we’ve changed is our culture of monitoring, adopting a mindset of continuous improvement. In this talk, we’ll take a look at what actions got us here, what we maybe should have done, and what we’ll never do again.