Communication-Efficient Computation Load Scheduling for Delay-Constrained Services

In this paper, we develop a dynamic communication-efficient computation load scheduling framework to complete the computation tasks with coded MapReduce considering arbitrary arrival and strict delay constraints over time-varying computing resource. Our goal is to minimize the communication load under the time-varying excess computing resources. We first reduce this problem to a task scheduling problem with exploiting the property of the computing repetition in the coded MapReduce framework. For the case that the full information about the available computing resource is known in advance, we obtain the optimal offline computation load scheduling algorithm. Guided by the optimal algorithm, we propose a dynamic online algorithm based on the predicted computing resource. Finally, our proposed algorithm is evaluated by simulation to demonstrate the superiority of the proposed algorithms over the conventional algorithms.