Scheduling Strategy for Deep Learning Training Jobs Based on Multi-Resource Interleaving

With the rapid advancement of deep learning (DL), different types of DL models exhibit significant variations in their resource utilization, including CPU, GPU, storage IO, and network IO. However, scheduling DL training jobs with diverse resource bottlenecks in resource-constrained clusters to minimize makespan remains challenging. Most existing studies primarily focus on GPU allocation, which limits their ability to effectively address the multi-resource demands of various model types. To address this challenge, we propose a multi-resource sharing model that leverages the staged iteration characteristics of DL jobs, thereby enabling parallel execution through temporal resource interleaving. Next, we propose a DL training job scheduling strategy based on the Dynamic Multi-Swarm Particle Swarm Optimization Algorithm with Genetic Algorithm Operators (DMPSO-GA). This approach dynamically partitions the swarm into subswarms based on particle fitness and enhances the algorithm's exploration capability by designing differentiated update strategies for particles at different evolutionary stages within the subswarms.