Ever since a JPEG created by the digital artist Beeple sold for more than $69 million in 2021, the worldwide obsession with NFTs (non-fungible tokens) that represent digital collectibles, art, and media has been growing.
A company at the forefront of the NFT world is the blockchain gaming studio Dapper Labs, which leverages blockchain to build addictive games (such as CryptoKitties), verify authentic digital collectibles, and run fan tokens for sports personalities and music artists. Their digital trading card system, NBA Top Shot, has scored roughly 19 million transactions.
Amid the fervor of “NFT Mania,” Dapper Labs has had to hyperscale to meet the global demand for crypto collectibles. Andrew Burian, the company’s principal SRE and engineering manager for the IT, security, and SRE teams, knew that the company wouldn’t be able to thrive without observability, so in 2019, he implemented an observability stack that includes Grafana Cloud, PagerDuty, Prometheus, Kubernetes, and Google Cloud Platform as its core monitoring tools.
To date, the Dapper Labs SRE team monitors more than 4 million active series, which translates to roughly 12 million data points per minute. With Grafana Loki for logs and Grafana Tempo for traces, Dapper Labs funnels all of its observability data through Grafana Cloud.
For Dapper Labs, the benefits of using Grafana Cloud have been wide-ranging.
For one thing, it has saved Burian’s team operational headaches around data retention because they now use Grafana to run visualization as well as deal with data warehousing. The time saved allows the six-person observability pod (which supports an engineering organization of more than 100 people total) to focus on bigger projects without having to worry about maintaining and upgrading every few months.
Using Grafana has also allowed the Dapper Labs team to engage in proactive problem-solving. In the past, customers would send performance issue reports long before monitoring caught anything. With their current instrumentation, they’re alerted to issues so quickly that they can even catch outages on Google Cloud before they’re publicly reported.
In addition to being more agile in alerting and monitoring, “the empowerment to the teams has been the best single result because that feeds into everything else we do,” says Burian. “There is more productivity and fewer mistakes.”
Grafana is heavily used by team members all over Dapper Labs. There are roughly 240 active users in Grafana, including project and product managers who want insight into the observability data. The company has an open editing culture, so engineers are welcome to jump in and tweak dashboards.
Even new hires tend to make meaningful contributions to the monitoring setup within their first two or three months.
It helps that “Grafana is visually appealing from the get-go,” says Burian. “It’s hard to make a dashboard that looks terrible.”
A Dapper Labs dashboard
Burian’s overall vision for observability at Dapper Labs is simple: “Real success is zeroing in on using our tooling and our observability to define really good customer-facing SLOs, make really impactful alerts and monitoring, and get what we need without just drowning ourselves in data,” he says.
Burian has “never been happier” with the company’s monitoring situation and plans to lean into tracing more with Grafana Tempo in the coming months and years. He adds, “Things can only get better from here.”