By Topic

An Efficient MPI Message Queue Mechanism for Large-scale Jobs

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

The purchase and pricing options are temporarily unavailable. Please try again later.
2 Author(s)
Zounmevo, J.A. ; Dept. of Electr. & Comput. Eng., Queen's Univ., Kingston, ON, Canada ; Afsahi, A.

The Message Passing Interface (MPI) message queues have been shown to grow proportionately to the job size for many applications. With such a behaviour and knowing that message queues are used very frequently, ensuring fast queue operations at large scales is of paramount importance in the current and the upcoming exascale computing eras. Scalability, however, is two-fold. With the growing processor core density per node, and the expected smaller memory density per core at larger scales, a queue mechanism that is blind on memory requirements poses another scalability issue even if it solves the speed of operation problem. In this work we propose a multidimensional queue traversal mechanism whose operation time and memory overhead grow sub-linearly with the job size. We compare our proposal with a linked list-based approach which is not scalable in terms of speed of operation, and with an array-based method which is not scalable in terms of memory consumption. Our proposed multidimensional approach yields queue operation time speedups that translate to up to 4-fold execution time improvement over the linked list design for the applications studied in this work. It also shows a consistent lower memory footprint compared to the array-based design.

Published in:

Parallel and Distributed Systems (ICPADS), 2012 IEEE 18th International Conference on

Date of Conference:

17-19 Dec. 2012