Diagnosing Performance Fluctuations of High-Throughput Software for Multi-core CPUs

Performance fluctuations are common in various software such as databases and software networking stacks. A fluctuation refers different performance (latency, throughput) for similar or identical data-items (e.g. requests, queries, packets) due to non-functional states such as cache warmth. While tail latency caused by fluctuations badly affect user experiences, diagnosing them is difficult as reproducing non-functional states in a controlled environment is not feasible. To this end, we estimate elapsed time of each function for each data-item individually to observe a single fluctuation occurrence online so that reproducing non-functional states is no longer needed. The issue is that instrumentation-based tracing methods are too heavy because a function takes a few micro seconds in high-throughput software systems for the multi-core age. We propose a hybrid approach of instrumentation and hardware-based sampling. It enables to diagnose performance fluctuations of high-throughput software systems with acceptable and adjustable overhead. Our evaluations show that it can clearly show a performance fluctuation that occurs by different cache-warmth in a sample application, and that it can be also applied to realistic software.