Deadline-Aware Flow Scheduling for AI Clusters with Heterogeneous Latency Requirements

Modern training clusters use a distributed approach to train deep learning jobs. In each communication round phase, distributed machine learning jobs generate multiple parallel data flows, which comprise a coflow group. In this literature, coflow scheduling has been widely studied and one common objective is to minimize the average job completion time. However, some jobs might have strict deadline, minimizing the deadline violation ratio is another important objective. We observe that competition among different jobs is a critical factor preventing jobs from meeting their deadlines. In this paper, we propose Allot, an online scheduler that combines admission control and flow scheduling. Allot accounts for inter-job competition when scheduling coflows, thereby minimizing the deadline violation ratio. Simulation results have demonstrated that Allot outperforms the benchmark solutions by more than 1.4× in meeting deadlines.