Encryption Techniques for Hadoop Distributed File System (HDFS)

The Hadoop Distributed File System (HDFS) was created with the express purpose of making the management of large data sets a breeze. The absence of something in data encryption methods in HDFS, however, presents serious security concerns. While the introduction of encryption methods has improved data security, traditional algorithms struggle when processing larger files. This study aims to provide a novel hybrid encryption technique for protecting massive datasets by fusing Cipher-Text Policy Attribute-Based Encryption (CP-ABE) and Advanced Encryption Standard (AES). Java-based open-source project that includes the MapReduce framework for data processing and the HDFS. It modifies many Portable Operating System Interface (POSIX) standards to provide streaming access to file system information. Initiated as a platform for the open-source web crawler Apache Nutch, HDFS has already expanded to include other similar programs. Authors have compared the method to the one that was present in the system, they have implemented it to protect HDFS, and performed experimental research to prove its performance. Further, the authors have tested the suggested HDFS technique by encrypting files of various sizes and compared it to other algorithms, such as AES and AES with One-Time-Pad (OTP). The HDFS method improves performance significantly when it relates to file encryption.