Evolution Towards Distributed Storage in a Nutshell

Distributed storage systems have greatly evolveddue to cloud computing upsurge in the past several years. Thedistributed file systems inherit many components fromcentralized ones and use them in a distributed manner. There aretwo ways to grow the storage capacity: by scaling-up or byscaling-out and growing the number of storage devices in astorage system. The growth of storage devices impose manychallenges related to interconnection protocols and topologies,error handling, data consistency, security and so on. In thisarticle we have studied how distributed and parallel storageshave evolved from direct connected storages in terms ofarchitecture, data management and organization and how thenew challenges imposed by data distribution have been solved.We have selected for studying several of the most representativedistributed storages solutions: Andrew File System, Google FileSystem, General Parallel File System, Lustre and Ceph. First, weemphasize how a generic distributed storage layout has inspiredfrom structured disk layout (Berkeley Fast File System). Second,we describe the evolution path of distributed storages from awide variety of perspectives, including: distributed units whichare moving from blocks to objects due to their undeniableadvantages or distribution methods that have evolved from listsmuch like inode mapping to deterministic hash functions likeRUSH or CRUSH. Third, the networks are evolving very fast interms of topologies and protocols. Using graph theory,researchers are continuously improving different aspects ofcluster networks. Fourth, storage security is a critical componentdue to the demand of storing sensitive data for long term, sharingit in a secure way and impacting as little as possible the systemperformance.