Scheduling deadline-constrained bulk data transfers to minimize network congestion

Tight coordination of resource allocation among end points in Grid networks often requires a data mover service to transfer a voluminous dataset from one site to another in a specified time interval. With flexibility at its best, the transfer can start from any time after its arrival, use any and even time variant bandwidth value, as long as it is completed before its deadline. Given a set of such tasks, we study the Bulk Data Transfer Scheduling (BDTS) problem, which searches for the optimal bandwidth allocation profile for each task to minimize the overall network congestion. We show that the multi-interval scheduling, which divides the active window of a task into multiple intervals and assigns bandwidth value independently in each of them, is both sufficient and necessary to attain the optimality in BDTS. Specifically, we show that BDTS can be solved in polynomial time as a Maximum Concurrent Flow Problem. The optimal solution attained is in the form of multi-interval scheduling with the number of intervals upper-bounded. Simulations are conducted over several representative topologies to demonstrate the significant advantage of optimal solutions.