OloOlo Delivers Faster Testing and Improved Collaboration with Grafana k6
Key takeaways
- k6 has become a vital component of Olo’s CI/CD pipeline by helping teams detect performance regressions before they reach production.
- k6 is based on JavaScript, so various teams can easily collaborate on the load testing process and PR reviews no longer contain changes to large XML files.
- With k6, performance testing has become a common practice, allowing Olo’s teams to continuously improve the platform and provide on-demand restaurant customers a seamless and enjoyable experience.
-–
Olo is a leading on-demand commerce platform powering the restaurant industry’s digital transformation. Millions of orders per day run on Olo’s enterprise SaaS engine, enabling brands to maximize the convergence of digital and brick-and-mortar operations. The Olo platform provides the infrastructure to capture demand and manage consumer orders from every channel. With integrations to over 100 technology partners, Olo customers can build digital experiences with the largest and most flexible restaurant commerce ecosystem on the market. Over 400 restaurant brands use Olo to grow digital sales, maximize profitability, and preserve direct consumer relationships.
The challenge
Load testing at Olo primarily involves two technical teams—software engineers in test (SETs) and software engineers (SWEs). For several years, the teams at Olo used a load testing stack that included JMeter to create test scripts and a CI/CD pipeline to orchestrate everything. The teams used the CI/CD pipeline along with NodeJS to run scripts via a well-known cloud load testing provider.
While onboarding a new restaurant brand, the Olo team noticed the brand was using k6 to test the integration on their end, and the engineers shared their scripts with them. The Olo teams discovered that the k6 CLI was significantly easier to automate and trigger load tests in the cloud than their current tool.
The SET team discovered that writing load tests in the Javascript-based k6 solution was a more pleasant experience than writing them in the Java-based JMeter application. A JavaScript solution meant that PR reviews wouldn’t consist of changes to large XML files allowing them to reuse modules and treat the load test code as a real test suite.
After learning more about k6, the teams realized that their existing load testing workflow wasn’t as fast, efficient, and collaborative as they needed.
The solution
The teams plugged k6 Cloud into their existing CI/CD pipeline, using it to test both public and private Olo HTTP APIs. For the private APIs, the teams run the tests from their own test networks. Then they send the results to k6 Cloud so that they can review them in its UI. k6 plays a vital role in Olo’s CI/CD pipeline by helping the teams detect performance regressions before they reach production.
Performance and reliability of our platform are of the utmost importance for Olo, our brands, and our partners. As a result, k6 has become an integral part of our test stack. k6 helps us quickly experiment with new ideas and verify that releases are production-ready.
Jake Travisano, Staff Software Engineer in Test at Olo
The teams use the k6 thresholds feature as a pass/fail indicator for release readiness. They test a wide range of components with thresholds, such as response times, CPU utilization, and database query latency. If any thresholds fail, the CI/CD jobs will alert the teams. In most cases, the SET team only needs to do a quick review of the results in k6’s UI to find the problem.
The teams use the visualization and analysis features as a focal point for collaboration, often referring to detailed charts and reports to discuss test results and action plans. The teams naturally share technical knowledge, domain knowledge, and platform ideas while collaborating. As more engineers gain experience with k6, performance testing has become a regular activity for teams—whether running tests daily or working a performance testing plan into the development of new features and projects.
The results
With k6, various engineering teams can quickly write, modify, and execute load testing scripts, which has helped performance testing become a common practice at Olo. This common practice has allowed the teams to continuously improve Olo’s platform and provide restaurant customers a seamless and enjoyable experience.
Our usage of k6 has played a critical role in helping us to push our system to its limits, identify its breaking points, and explore how it would behave in unexpected circumstances. k6 has also become an invaluable tool for experimenting with performance optimizations.
Justin Rusbatch, Staff Software Engineer, Olo
Collaboration among teams has improved significantly— SETs, SWEs, and database administrators (DBAs) regularly pair up to plan and run tests then discuss and analyze results. In addition, SETs have expanded collaboration efforts through k6. They use k6 on occasion in collaboration with their site reliability engineering (SRE) team to help train new incident responders.
With a shared philosophy around performance testing and better collaboration among teams, confidence in production deployments has increased significantly among Olo’s teams.
