What is continuous profiling?
This post was originally published on pyroscope.io and was written by Uchechukwu Obasi and Ekaterina Patrice. Grafana Labs acquired Pyroscope in 2023.
Coming from a background as frontend developers at Grafana, we are no strangers to open source performance monitoring. We were part of a team that was responsible for the overall user experience of Grafana and performance was one of the key considerations. Along the line, we learned about a debugging technique known as profiling for monitoring application performance and fell in love ever since.
What is continuous profiling?
“Profiling” is a dynamic method of analyzing the complexity of a program, such as CPU utilization or the frequency and duration of function calls. With profiling, you can locate exactly which parts of your application are consuming the most resources. “Continuous profiling” is a more powerful version of profiling that adds the dimension of time. By understanding your system’s resources over time, you can then locate, debug, and fix issues related to performance.
As frontend developers, our experience with profiling was limited to the browser. However, we discovered a new pattern of profiling that seems exciting – continuous profiling. Similar to how you use the profiler in the dev console to understand frontend performance issues, continuous profiling allows you to profile servers from various languages 24/7 and be able to understand resource usage at any particular time.
While continuous profiling is new to many, the concept is actually relatively old. In 2010, Google released a paper titled “Google Wide profiling: A continuous profiling infrastructure for data centers” where they make the case for the value of adding continuous profiling to your applications.
Industry traction for continuous profiling
Since then, many major performance monitoring solutions have joined them in releasing continuous profiling products. As time has gone on, the continuous profiling space has been getting increasingly popular as various companies/VCs are more frequently making major investments in continuous profiling to keep up with demand for this type of monitoring.
It’s not just major cloud providers and venture capitalists who are excited about continuous profiling; industry leaders are super pumped about continuous profiling, too!
"On The State Of Continuous Profiling" good writeup by @mhausenblas - probably time we update the cloud native landscape to add this category of observability https://t.co/5NCjYbIU44
— Chris Aniszczyk (@cra@macaw.social) (@cra) December 23, 2021
Chris Aniszczyk, CTO of the Cloud Native Computing Foundation (CNCF), agrees that continuous profiling is an integral part of the observability stack.
What signal types are you capturing in your Kubernetes cluster (infra or workload level)? #Kubernetes #monitoring
— Michael Hausenblas (@mhausenblas) November 20, 2021
This poll by Michael Hausenblas, solution engineering lead at AWS, is good evidence that continuous performance profiling is becoming a part of the culture for most engineering teams.
I suspect that we will see tons of submissions about profiling next year as parca, pyroscope and pixie gets more traction.
— Carl Bergquist (@CarlBergquist) February 6, 2022
Even Carl Bergquist, Principal Engineer at Grafana Labs, predicts that the popular FOSDEM conference will see more profiling talks moving forward.
Simply put, continuous profiling is the future of application performance assessment.
Continuous profiling using Pyroscope
At Pyroscope, we actually profile our own servers using Pyroscope and have found many cases where profiling has saved us a lot of time and money by identifying performance issues. That being said, at any organization, understanding resource utilization and adjusting to allocate it efficiently can be the difference between a healthy company with happy end-users (and employees) or a chaotic one where everyone is running around putting out fires and end-users are suffering.
There is no better tool than continuous profiling to help be the difference-maker in these situations. No matter what your job function is in your organization there are still very valuable benefits of having continuous profiling as part of your observability stack.
Pyroscope also provides you with multi-language support, ad-hoc profiling, profile sharing, and more!
What makes Pyroscope unique?
Pyroscope is open source which means that it’s easier to get started and try out than cloud providers’ profilers. Pyroscope’s custom storage engine is language-agnostic, so any profiling data, ranging from rbspy to eBPF to even new profilers such as Bloomberg’s Memray, can easily send data to Pyroscope and have it compressed, stored, and queried efficiently.
Pyroscope is also built to be easy to use by everyone. The Pyroscope UI, in particular, was designed to be super intuitive for anyone to play around with, analyze, and understand profiles.
Whether you are operating at a small scale or enterprise-level, the power of continuous profiling can quickly and easily reduce infrastructure costs and improve scalability. With Pyroscope, you can build high-performance applications that increase customer satisfaction and put value back into your business.
How to get started with Pyroscope
One of the easiest ways to get started with continuous profiling is to install Pyroscope. To learn how to use continuous profiling in your code or application, we suggest you run the most relevant, Dockerized example from the examples folder on GitHub or you can take a look at the installation guide here.
If you need any help getting started, join the community Slack or follow Pyroscope on GitHub!
