Coflow Deadline Scheduling via Network-Aware Optimization

Several cloud or data-parallel applications involve coflow scheduling, which controls a set of flows under the same semantic meaning with a common goal. Due to such common goal, optimizing each flow using the standard flow scheduling approaches does not necessarily lead to good coflow performance. Therefore, numerous methods and systems are proposed to focus on coflow scheduling problem. However, the current coflow scheduling designs are based on simple heuristics or observations. The absence of the optimum makes it hard to adjudge their absolute effectiveness. In this work, we derive the optimal solution to the coflow deadline satisfaction problem (CDS), which maximizes the number of satisfied coflow deadlines, from a mixed integer linear program formulation. We further show that CDS is not only NP-hard to solve but also intractable to approximate with a fixed approximation ratio (unless P=NP). As such, the use of heuristics is justified. We then develop optimization-based methods to approach the problem offline and online. The proposed methods are simulated and compared against the optimum, along with some state-of-the-art designs, and the results suggest that our methods are much closer to the optimum than the existing ones, especially when we have more room to schedule.