Analysis of Resources Scheduling for DL Workloads on Multi-tenant GPU Clusters

Deep learning (DL) has already played more and more crucial roles in various fields, such as medicine, service, manufacturing, etc. The high-performance deep learning training tasks mainly rely on the GPU resource. Thus, an increasing number of large-scale technology-centric companies set up multi-tenant GPU clusters to train the DL jobs. Yet, considering the characteristics of DL jobs and GPUs' architecture, the efficiency issues are still the bottleneck for the current multi-tenant GPU clusters and even the improvement of DL: users cannot share resources with the fine-grained standard since GPU is a kind of single resource; moreover, there are the novel and comprehensive requirements with the development of distributed DL jobs. This review introduces the development route of popular GPU cluster schedulers and scheduling frameworks in detail, clarifying how they reduce training time, improve resource utilization, and make sure sharing safety. In addition, based on these analyses, we also propose two focus points as future guidelines: locality constraints and contention for shared resources. In the remaining paper, we point that the elastic and seamless migration strategies for future research.