Multi-User Linearly-Separable Distributed Computing

In this work, we explore the problem of multi-user linearly-separable distributed computation, where
$N$
servers help compute the desired functions (jobs) of
$K$
users, and where each desired function can be written as a linear combination of up to
$L$
(generally non-linear) subtasks (or sub-functions). Each server computes some of the subtasks, communicates a function of its computed outputs to some of the users, and then each user collects its received data to recover its desired function. We explore the computation and communication relationship between how many servers compute each subtask vs. how much data each user receives. For a matrix
$\mathbf {F}$
representing the linearly-separable form of the set of requested functions, our problem becomes equivalent to the open problem of sparse matrix factorization
$\mathbf {F} = \mathbf {D}\mathbf {E}$
over finite fields, where a sparse decoding matrix
$\mathbf {D}$
and encoding matrix
$\mathbf {E}$
imply reduced communication and computation costs respectively. This paper establishes a novel relationship between our distributed computing problem, matrix factorization, syndrome decoding and covering codes. To reduce the computation cost, the above
$\mathbf {D}$
is drawn from covering codes or from a here-introduced class of so- called ‘partial covering’ codes, whose study here yields computation cost results that we present. To then reduce the communication cost, these coding-theoretic properties are explored in the regime of codes that have low-density parity check matrices. The work reveals — first for the commonly used one-shot scenario — that in the limit of large
$N$
, the optimal normalized computation cost
$\gamma \in (0,1)$
is in the range
$\gamma \in \left({H_{q}^{-1}\left({\frac {\log _{q}(L)}{N}}\right), H_{q}^{-1}(K/N)}\right)$
— where
$H_{q}$
is the
$q$
-ary entropy function — and that this can be achieved with normalized communication cost that vanishes as
$\sqrt {\log _{q}(N)}/N$
. The above reveals an unbounded coding gain over the uncoded scenario, as well as reveals the role of a certain functional rate
$\log _{q}(L)/N$
and functional capacity
$H_{q}(\gamma)$
of the system. In the end, we also explore the multi-shot scenario, for which we derive bounds on the computation cost.