RAPID: A Fast Data Update Protocol in Erasure Coded Storage Systems for Big Data

Erasure codes are nowadays used extensively in distributed storage systems that handle big data, since they offer significant fault tolerance with low storage overhead. Even though erasure coded systems are space efficient, these involve higher network bandwidth and computational complexity in their operations. In this paper, we present RAPID, a protocol for fast data updates, which works by choosing a subset of code blocks for updates and adapts the strength of the subset based on the predicted number of failures. The proposal uses a prediction based heuristic in which the set of failures that may happen in the near future is represented as a function of past failures. A hybrid protocol that uses both locking and buffering mechanisms is adopted in the solution to maintain the consistency on the data and code blocks updates. Our experimental results demonstrate improvement in the performance of data updates by 30% and the failure prediction mechanism proposed shows an accuracy of 80%.