Coded distributed computing for system with stragglers

Distributed computing is an important topic in the field of wireless communications and networking, and its high efficiency in handling large amounts of data is particularly noteworthy. Although distributed computing benefits from its ability of processing data in parallel, the communication burden between different servers is incurred, thereby the computation process is detained. Recent researches have applied coding in distributed computing to reduce the communication burden, where repetitive computation is utilized to enable multicast opportunities so that the same coded information can be reused across different servers. To handle the computation tasks in practical heterogeneous systems, we propose a novel coding scheme to effectively mitigate the “straggling effect” in distributed computing. We assume that there are two types of servers in the system and the only difference between them is their computational capabilities, the servers with lower computational capabilities are called stragglers. Given any ratio of fast servers to slow servers and any gap of computational capabilities between them, we achieve approximately the same computation time for both fast and slow servers by assigning different amounts of computation tasks to them, thus reducing the overall computation time. Furthermore, we investigate the information-theoretic lower bound of the inter-communication load and show that the lower bound is within a constant multiplicative gap to the upper bound achieved by our scheme. Various simulations also validate the effectiveness of the proposed scheme.