An efficient autoscaling of Hadoop clusters in public cloud

The cloud computing applications has key building blocks in distributed file system based on Map Reduced programming. Nodes serve computing features and storage. A file is divided into chunks and that can be allocated in the nodes. As files can be created, moved, updated and deleted. This leads to load imbalance in the file system. It is necessary to distribute the file chunks uniformly across the nodes. The existing production systems heavily depend on the central node for reallocation. This dependency is not sufficient for large scale environment and leads to performance degradation and single point of failure. Hence, in this work a frame work is designed to balance overload and shown how it outperforms the conventional distributed system and competes a better solution. The proposed work is implemented with Amazon web service led is a public cloud. It is determined that heuristics sets prevent overload in the system. This framework will gather heartbeats, performance metrics and based on the data start more instances. Initially the Hadoop starts using the new instances and a job that enters the environment gets executed faster.