Approaches to deployment of Hadoop on cloud platforms: Analysis and research issues

Hadoop is an established data processing engine used for “Big Data” processing in cloud computing environment due to its ease-of-use, high scalability, cost effectiveness and fault tolerance. Technology businesses such as Amazon, Facebook, Yahoo, Microsoft, and Twitter, have created large datasets leading to increased requirements for data storage and processing power for dataset analysis. The main challenge is the significant volume of data that needs to be handled by cloud applications. To overcome this problem, Hadoop clusters are increasingly deployed on large number of virtual machines in cloud data centers which leads to increase in productivity and decrease in consumption of energy and resource. In this paper, we discuss various approaches for deploying Hadoop in the cloud and identify the challenges in deploying virtual Hadoop clusters. We analyze the research issues involved and performance evaluations presented in various research papers based on energy consumption, scalability and response time using different benchmarks for different virtualization technologies with deployed Hadoop.