Efficient Online Scheduling for Coflow-Aware Machine Learning Clusters

Distributed machine learning (DML) is an increasingly important workload. In a DML job, each communication phase can comprise a
coflow
, and there are dependencies among its coflows. Thus, efficient coflow scheduling becomes critical for DML jobs. However, the majority of existing solutions focus on scheduling single-stage coflows with no dependencies. While there are a few studies schedule dependent coflows of multi-stage jobs, they suffer from either practical or theoretical issues. Motivated by this situation, we study how to schedule dependent coflows of multiple DML jobs to minimize the total JCT in a shared cluster. We present a formal mathematical formulation for this problem and prove its NP-hardness. To solve this problem without job size information, we present an online coflow-aware optimization framework called
Parrot
. The core idea in
Parrot
is to infer the job with the shortest remaining processing time (SRPT) each time and dynamically control the inferred job's bandwidth based on how confident it is an SRPT job while being mindful of not starving any other job. Specifically, in the design of
Parrot
, we present a least per-coflow attained service (LPCAS) policy to infer the SRPT job. We further propose a dynamic job weight assignment mechanism and a linear program (LP) based weighted bandwidth scaling strategy for sharing bandwidth among DML jobs. We have proved that
Parrot
algorithm has a non-trivial competitive ratio. The results from large-scale trace-driven simulations further demonstrate that our
Parrot
can reduce the total JCT by up to 58.4 percent, compared to the state-of-the-art Aalo solution.