By Topic

Design Dynamic Data Allocation Scheduler to Improve MapReduce Performance in Heterogeneous Clouds

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

3 Author(s)
Shin-Jer Yang ; Dept. of Comput. Sci. & Inf. Manage., Soochow Univ., Taipei, Taiwan ; Yi-Ru Chen ; Yung-Ming Hsieh

This paper conducts a thorough research on one of the critical technologies in cloud computing, MapReduce programming model. Some of past research results showed that their methods can be executed through allocating identical tasks to each cloud node for enhancing MapReduce performance. However, such allocations are not applicable for the environment of heterogeneous cloud. Due to the different computing power and system resources between the nodes, such uniform distribution of tasks will lower the performance between nodes, and hence this paper makes improvement on the original speculative execution method of Hadoop and LATE Scheduler by proposing a new scheduling scheme known as Dynamic Data Allocation Scheduler (DDAS). DDAS adopts more accurate methods to determine the response time and backup task that affect the system, which is expected to enhance the success ratio of backup tasks and thereby to effectively increase the system ability to respond. Three different simulation experiments are performed and the using of DDAS scheme proves that that DDAS can reduce 30%, 18% and 21% of execution time relative to Hadoop. Also, the DDAS shows a more accurate speculative execution and reasonable allocation of backup tasks. Hence, DDAS can effectively enhance the performance of MapReduce processing in heterogeneous Cloud environment.

Published in:

e-Business Engineering (ICEBE), 2012 IEEE Ninth International Conference on

Date of Conference:

9-11 Sept. 2012