Multi-level Adaptive Execution Tracing for Efficient Performance Analysis

Troubleshooting system performance issues is a challenging task that requires a deep understanding of various factors that may impact system performance. This process involves analyzing trace logs from the kernel and user space using tools such as ftrace, strace, DTrace, or LTTng. However, pre-set tracing instrumentation can lead to missing important data where not enough components of the system include observability coverage. Also, having too much coverage may result in unnecessary noise in the data, making it extremely difficult to debug. This paper proposes an adaptive instrumentation technique for execution tracing, which dynamically makes decisions not only for which components to trace but also when to trace, thus reducing the risk of missing important data related to the performance problem and increasing the accuracy of debugging by reducing unwanted noises. Our case study results show that the proposed method is capable of handling tracing instrumentation dynamically for both kernel and application levels while maintaining a low overhead.