Visualizing the Linux Kernel Performance with FlameGraph AR

In this challenge, we explore the evolution of the Linux kernel’s performance during compilation by comparing versions 5.19.17 and 6.14 through sampling-based CPU profiling. We collect profiling data using perf, transform into Chromecompatible .cpuprofile format, and analyze through a novel spatial visualization called FlameGraph AR.FlameGraph AR extends traditional flamegraphs beyond the limitations of IDE panels and conventional screens by rendering visualizations with augmented reality on a Microsoft HoloLens 2 device. By offloading the flamegraph to physical space, the FlameGraph AR tool enables developers to walk through wide and deeply nested call stacks, examine function frames through gesture-based interactions, and gain spatial awareness of the runtime behavior of a software system.In effect, we found immersive visualization especially valuable for analyzing architectural changes between the two kernel versions. We found that version 6.14 exhibits a significantly higher number of samples in several functions, such as native_write_msr, indicating intensified low-level CPU interactions. In addition, functions such as intel_pmu_enable_all and x86_pmu_enable also increased in frequency, suggesting increased reliance on performance monitoring. The stack depth analysis revealed that certain functions in version 6.14, including fpregs_assert_state_consistent and account_user_time, appear at significantly deeper levels than in earlier versions. Indeed, some reach the maximum stack trace depth of the profiling tool. The results indicate a growth in both modularity and the depth of instrumentation within the kernel execution paths.Multiple performance changes become visible and interactive with Flamegraph AR. For example, time-consuming functions show up as wide frames that span over desks or walls, and deep call stacks are explored physically by approaching or gazing upward. By mapping performance traces into the spatial domain, our tool provides a compelling method for understanding systemic evolution in large-scale software like the Linux kernel.Video URL: https://vimeo.com/1092935027/7d09676a83