Magicube: High Reliability and Low Redundancy Storage Architecture for Cloud Computing

High reliability, high performance and low (space) cost are three important priorities for storage systems. However, it's hard for a cloud storage system to fit them all because they are conflict with each other. Currently, widely used cloud storage systems, such as Google File System (GFS), Hadoop Distributed File System (HDFS) and Amazon's Simple Storage Service (S3), can well meet with high reliability and high performance, but suffer from a huge extra space overhead because of their multi-replication policy. In this paper, we introduce Magicube - a high reliable and low redundancy storage architecture for cloud computing. With only one replica in HDFS, and an (n, k) algorithm for fault-tolerant, it satisfies both low space overhead and high reliability simultaneously. By executing the fault-tolerant process in the background, the performance of Magicube is also good. According to our experiments' result, Magicube can work well for batch processing jobs.