Novel Decentralized Security Architecture for the Centralized Storage System in Hadoop using Blockchain Technology

Big Data is huge in volume, diverse in information, and growing at flourishing rates. The major distributed file systems in the current market in Big Data Analysis includes Apache Hadoop, Storm, Cassandra, Flink, Cloudera, and many more. Hadoop is an open-source framework divided into Hadoop Distributed File System (HDFS) and Map-Reduce. Hadoop plays a leading role in storing and processing Big Data in contemporary society as it is cost- effective and can manage large volumes of data in low-cost commodity hardware. HDFS is a type of Data Warehouse which is scalable and has fast access to the information. Metadata is the information about the data, such as which block is storing on what datanode, how many replications are that particular block has, and on which datanodes those replications reside. In HDFS, this metadata is stored at a fixed place in namenode, and attackers can access the metadata and modify it without notice. Also, the metadata is mutable, which means that the attacker can erase his presence easily. To resolve this issue, in this paper we have provided a mechanism using blockchain technology that follows a decentralized architecture against the centralized architecture followed by HDFS. Hyperledger Fabric (HLF) is the blockchain proposed to be effective and trusted for such a purpose. HLF is a private blockchain with a distributed immutable ledger. The metadata will be stored in the ledger. If an attacker tries to modify the data, he cannot erase his presence as the ledger is immutable, unlike HDFS. Further, the work proposed in this paper can be extended in real-time HDFS with the secure ledger and multiple nodes.