By Topic

CloudBATCH: A Batch Job Queuing System on Clouds with Hadoop and HBase

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Chen Zhang ; David R. Cheriton Sch. of Comput. Sci., Univ. of Waterloo, Waterloo, ON, Canada ; De Sterck, H.

As MapReduce becomes more and more popular in data processing applications, the demand for Hadoop clusters grows. However, Hadoop is incompatible with existing cluster batch job queuing systems and requires a dedicated cluster under its full control. Hadoop also lacks support for user access control, accounting, fine-grain performance monitoring and legacy batch job processing facilities comparable to existing cluster job queuing systems, making dedicated Hadoop clusters less amenable for administrators and normal users alike with hybrid computing needs involving both MapReduce and legacy applications. As a result, getting a properly suited and sized Hadoop cluster has not been easy in organizations with existing clusters. This paper presents Cloud BATCH, a prototype solution to this problem enabling Hadoop to function as a traditional batch job queuing system with enhanced functionality for cluster resource management. With Cloud BATCH, a complete shift to Hadoop for managing an entire cluster to cater for hybrid computing needs becomes feasible.

Published in:

Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on

Date of Conference:

Nov. 30 2010-Dec. 3 2010