By Topic

An Adaptive Middleware Framework for Optimal Scheduling on Large Scale Compute Clusters

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

4 Author(s)
Gosney, A. ; Pacific Northwest Nat. Lab., Richland, WA, USA ; Miller, J.H. ; Gorton, I. ; Oehmen, C.

In production multi-user high-performance (HPC) batch computing environments, wait times for scheduled jobs are highly dynamic. For scientific users, the primary measure of efficiency is wall clock time-to-solution. In high throughput applications, such as many kinds of biological analysis, the computational work to be done can be flexibly scheduled taking a longer time on a small number of processors or a shorter time on a large number of processors. Therefore the capability to choose a platform at run-time based on both processing capabilities and availability (lowest wait time) would be attractive. The goal of our work was to create an adaptive interface to HPC systems that dynamically reschedules high-throughput calculations in response to fluctuating load, optimizing for time-to-solution. This was done by implementing middleware functionality to (1) monitor the resource load on a given compute cluster, (2) generate a plan, checking on the applicability of the plan with the defined goals and (3) adaptively choosing the optimal job dimensions (number of processors and wall-clock time) to provide the best time-to-solution results.

Published in:

Information Technology: New Generations (ITNG), 2011 Eighth International Conference on

Date of Conference:

11-13 April 2011