ESTHER: Application-First Hardware-Level QoS-Enforcement for Cloud Native Environments

Recent advances in multi-core chip technology have enabled the dynamic tuning of shared memory resources, such as last-level cache and memory bus bandwidth. However, despite proven performance benefits, the complexity of effectively utilizing these hardware-level QoS enforcement features has limited their adoption in real-world cloud computing environments. In this paper, we introduce ESTHER, a novel approach to autonomously fine-tune QoS enforcement features in cloud environments using extremum seeking control, focusing on applications needs and operator ease-of-use. We demonstrate that ESTHER effectively maintains latency-critical workload SLOs and rapidly resolves any infringements by prioritizing shared memory resources. Such fast node-level resolution of SLO violations ensures that costly cluster-level scaling events may be avoided. Furthermore, ESTHER improves best-effort job throughput without impacting latency-critical workloads, achieving performance gains without utilizing workload profiling or prior knowledge of system dynamics.