Block Allocation of Systematic Coded Distributed Computing in Heterogeneous Straggling Networks

Recently, coding techniques have been introduced in distributed computing systems, i.e., coded distributed computing (CDC), to alleviate the heterogeneous straggler effect. However, these techniques bring about additional decoding latency impacting on the task completion time. In this paper, we study the issues of load allocation and latency analysis of systematic CDC in heterogeneous computation and communication straggling networks (HCCSNs). In order to exploit the partial works completed by straggling workers, we use the method of block division to accelerate the sub-tasks' results returning from all workers. Moreover, we attempt to leverage the systematic MDS code, which needs fewer decoding operations, to reduce the decoding latency, but it requires prior determining of the systematic blocks and the parity blocks on the master not on the workers. Therefore, in order to minimize both of the execution (communication and computing) latency and decoding latency, we propose two algorithms, i.e., greedy-based binary search algorithm (GBSA) and proportional systematic block allocation (PSBA), to obtain the optimal numbers of blocks and systematic blocks assigned to each worker, respectively. Simulation results are presented to show that GBSA and PSBA outperforms other conventional block allocation schemes in both execution latency and decoding latency with various straggling parameters.