Efficient Big Data Storage and Retrieval in Distributed Architecture using Blockchain and IPFS

In current distributed file system, Big data storage suffers from security issues, single point of failure and scalability problems. With the popularity of bitcoin and crypto world, Blockchain has emerged as new decentralized framework. Now, Blockchain is also used as distributed storage framework for storing large data. When the volume of data increases, blockchain suffers from storage overhead and scalability issues. To solve this problem, we propose blockchain based distributed big data storage using IPFS(Inter-Planetary File System). IPFS reduces the storage overhead by storing small pieces of large file in multiple nodes and collectively generating hash of the file. Further elasticsearch is also used for efficient query retrieval. Overall,the above architecture of blockchain, IPFS and Elasticsearch enhances the big data storage, search latency and precision. It also provides access control mechanism which allows only authorized user to access the files. Thus this framework provides efficient data storage, security and its retrieval.