ANOLE: A Profiling-Driven Adaptive Lock Waiter Detection Scheme for Efficient MP-guest Scheduling

In today's data center, there is a growing virtualization evolving trend to consolidate multiple servers into a single physical system. New architecture design that couples more and more cores into one processor furthers this trend. However, virtualization also poses new challenges such as lock holder preemption. In this work, we first demonstrate that lock holder preemption could bring dramatic performance degradation in virtualization environment. Then we propose ANOLE, a runtime adaptive lock waiter detection approach for lock holder preemption overhead reduction of MP guests. It leverages the modern hardware feature without any modification to spin lock implementation. ANOLE implements a hyper visor framework to preempt virtual CPUs adaptively and a user agent for guest spin lock profiling on KVM. We present in-depth performance evaluation under different scenarios, covering simple OS workloads, SPECvirt, and windows guest workloads. The experiment results demonstrate the solid performance benefit of ANOLE, it brings up to 50% performance improvement under different usage scenarios, with better lock waiter detection and load balance.