Improving Efficiency and Performance Through Faster Scheduling Mechanisms

Today's high-performance applications demand microsecond-scale tail latencies and high request rates from operating systems, and most applications handle loads that have high variance over multiple timescales. Achieving these goals in an efficient way, where multiple applications must share the resources within each machine, is a difficult problem for today's operating systems. Their primary shortcoming is that they adjust resource allocations too slowly—over 10s or 100s of milliseconds. This leaves them vulnerable to extreme spikes in tail latency under shifts in resource demand while, at the same time, leaving resources underutilized. In this talk, I will present Caladan, a new approach to CPU scheduling that can achieve significantly better tail latency, throughput, and resource utilization through a collection of control signals and policies that rely on faster core allocation. Caladan consists of a centralized scheduler core that actively manages resource contention in the memory hierarchy and between hyperthreads, and a kernel module that bypasses the standard Linux Kernel scheduler to support microsecond-scale monitoring and placement of tasks. Caladan can maintain nearly perfect performance isolation between colocated applications with phased behaviors, and it lays the groundwork toward applying faster scheduling techniques to a broader range of systems problems.