Implementation of GPU Scheduling Method for Kubernetes

Kubernetes is an open-source system used for automating the deployment, scaling, and management of containerized applications, and is a core component of the Power Cloud platform. The default scheduling mechanism in Kubernetes primarily evaluates node scores based on CPU and memory utilization. While this approach assesses the resource consumption of nodes, it fails to incorporate multidimensional metrics for evaluating nodes and adapting to the complexity of GPU resources. Consequently, it fails to ensure appropriate GPU resource allocation for specific workloads within the Power Cloud platform. To address the complexity and high-stability requirements of the Power Cloud platform, this study proposes a GPU resource scheduling scheme based on Prometheus. This scheme utilizes Prometheus as a data collection tool to gather multidimensional metrics such as GPU usage, disk utilization, and network traffic across nodes in the cluster, using these metrics as additional scoring criteria. Experimental results demonstrate that the proposed scheduler outperforms the default scheduler, achieving better resource allocation and load balancing.