Chaotic Leader Particle swarm optimization (CL-PSO) for efficient data storage in a cloud-based Hadoop distributed file system (HDFS)

There are numerous challenges associated with storing data in the Hadoop Distributed File System (HDFS) that must be overcome to ensure reliable and efficient operation. Effectively monitoring data replication across numerous nodes to decrease storage overhead and assure fault tolerance is a major challenge. Furthermore, it is essential to address problems of individual Data Nodes while preserving the consistency of the data. Maintaining enough data replication in the data nodes and balancing cluster resources are necessary for optimizing data locality and performance. Chaotic leader Particle Swarm Optimization (CL-PSO) is a stochastic optimization technique that is inspired by the social behavior of bird flocking is used to optimize the distribution of data blocks efficiently ensuring the data availability and reduced latency across a cluster of data nodes in a HDFS The simulation outcomes demonstrate that the storage approach utilizing CL-PSO effectively identifies optimal data nodes and places data blocks with an ideal replication factor. Consequently, this optimization leads to decreased network bandwidth usage within the rack, resulting in reduced read and write latency. Moreover, it enhances data availability and delivery ratio % by comparing the different replication factors of data block storage within the data nodes of the racks.