General Coded Computing in a Probabilistic Straggler Regime

Coded computing has demonstrated promising results in addressing straggler resiliency in distributed computing systems. However, most coded computing schemes are designed for exact computation, requiring the number of responding servers to exceed a certain recovery threshold. Additionally, these schemes are tailored to highly structured functions. Recently, new coded computing schemes for general computing functions, where exact computation is replaced with approximate computation, have emerged. In these schemes, the availability of additional results leads to more accurate estimations of the computational tasks. This flexibility introduces new questions that need to be addressed. This paper considers a practically important scenario in the context of general coded computing, where each server may become a straggler with probability $p$, independently of others. We theoretically analyze the approximation error of two existing general coded computing schemes: Berrut Approximate Coded Computing (BACC) and Learning-Theoretic Coded Computing (LeTCC). Under the probabilistic straggler configuration, we demonstrate that the average approximation error for BACC and LeTCC converges to zero at rates of at least $\mathcal{O}\left(\log _{1 / p}^{4}(N) \cdot N^{-2}\right)$ and $\mathcal{O}\left(\log _{1 / p}^{3}(N) \cdot N^{-3}\right)$, respectively. This is perhaps surprising, as earlier results do not indicate convergence when the number of stragglers scales with the total number of servers $N$. However, in this case, despite the average number of stragglers being $N p$, the independence of servers in becoming stragglers allows the approximation error to converge to zero. These theoretical results are validated through experiments on various computational tasks, including deep neural networks.