The Auto-configurable LDPC Codes for Distributed Storage

The current distributed storage systems mainly rely on data replication to ensure certain level of data availability and reliability. A recent trend is to introduce erasure codes into the distributed storage. Inspired by the RAID system, early attempts have been focused on designing Reed-Solomon (RS) based solutions and other block codes including Low Density Parity Check (LDPC) codes. This paper investigates in details about the usage of the system resources when different configurations of LDPC codes are applied to distributed storage systems. Auto-configurable LDPC-based method is proposed and integrated with the Hadoop system with various configurations of LDPC codes. Simulations show great improvement in terms of encoding and repairing latencies compared with Reed-Solomon. Simulations also show that trade-off in terms of different system resources is achieved through different configurations.