By Topic

Light-weight Cloud Job Management System for Data Intensive Science

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Haehyun Kim ; Dept. of Infrastruct. Technol. Dev., Korea Inst. of Sci. & Technol. Inf., Daejeon, South Korea ; Jaegyoon Hahm

The size of datasets generated and analyzed by scientific applications are increasing abruptly. Computing resource configuration required in data intensive science is being changed rapidly as well. Scientists have dynamic computing environment needs, which can be supported by cloud computing. We propose a light-weight cloud-based job management system to execute data analysis in distributed systems. In contrast to the traditional job managers based on scheduler with push mode, light-weight job coordinator is designed to process data request by pull mode in the proposed job management system. The agents in distributed virtual machine instances request the data URLs to be analyzed through the coordinator and dispatch the data from repositories. Using cloud service for provisioning of computing resources offers scalable, effective, and possibly massive computing environment. Hence the designed job management system is suitable for data intensive science requiring high throughput computing resources and reasonable I/O performance for big scientific data. In order to find out requirements and experiment system functions, we applied one of astronomy applications.

Published in:

Utility and Cloud Computing (UCC), 2011 Fourth IEEE International Conference on

Date of Conference:

5-8 Dec. 2011