Shared Resource Orchestration Extensions for Kubernetes to Support Real-Time Cloud Containers

The improvements in network latency and the advent of Edge computing have inspired industries to explore providing Real-time (RT) applications as cloud-based services and benefit from the availability, scalability, and efficient hardware resource utilization of clouds. It is crucial to improve the entire stack, including the applications’ containerization, container deployment, and orchestration across nodes to host RT applications in the cloud. However, state-of-the-art container orchestrators, e.g., Kubernetes (K8s), and the underlying Linux and containerization layer ignore orchestration and management of shared resources (e.g., memory bandwidth, cache); thus, rendering them unsuitable for RT use cases due unpredictability as a result of shared resource contention. We propose K8s extensions inspired by existing RT resource management frameworks to the underlying Linux kernel and containerization layer of each node for shared resource monitoring to help K8s maintain a cloud-wide view and allocate and dynamically orchestrate shared resources to enforce the guarantees required by the RT containers. Additionally, as a proof-of-concept, we design and implement (1) new K8s shared resource orchestration extensions to support memory bandwidth and last-level cache allocation and (2) a shared-resource controller in Linux based on a new algorithm to combine approximate but throttling-free memory bandwidth allocation by simple and efficient hardware controllers (e.g., Intel MBA) together with strict but pessimistic guarantees offered by software budget allocation and throttling (e.g., Memguard). We performed experiments to evaluate and demonstrate the newly implemented extensions on server-grade hardware.