Robust Distributed Storage of Residue Encoded Data

We consider a problem where a physical quantity is repeatedly measured by replicated devices, yielding a stream of numerical data. Data are stored within the measuring devices and sporadically retrieved by a user. To avoid data losses due to large data streams with insufficient memory, the data are split into fragments, each of which is a compressed encoding of a number in the stream, and different fragments are stored in different, replicated devices. The devices are not allowed to communicate with each other, and they produce the local streams of fragments from independent measurements. Given the independence of measurements, the fragments are corrupted by independent errors, which are likely to be small integers, although errors of unbounded magnitude may also occur due to failures or to interferences. As devices may fail, or communication may be unreliable, the user may be unable to download fragments from some of the replicated devices, leading to fragment erasures. Our approach to the problem is to encode the data in a Residue Number System with Nonpairwise-Prime Moduli, named D-RNS-NPM. With
n
moduli and
n
residue digits, every replicated device is tied to a different modulus, with which it produces and stores a residue digit (i.e., a fragment) from the local measurement. Assuming an upper bound
z
, with
z
<;
n
, to the number of erasures, we show that the D-RNS-NPM guarantees the reconstruction of any number from a subset of at least
n
-
z
fragments. If fragments bear errors, whose magnitude is unrestricted for at most one error and upper bounded by a small δ for the others, reconstruction is within an approximation of ±δ, and this property is retained when errors cannot be detected due to the unbounded error multiplicity. The time complexity of the decoding algorithm is polynomial. This problem appears to be relevant in wireless sensor networks, and an application in this area is envisioned.