By Topic

An efficient MPI-IO for noncontiguous data access over InfiniBand

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

3 Author(s)
Ding-Yong Hong ; Dept. of Comput. Sci., Nat. Tsing Hua Univ., Hsinchu, Taiwan ; Ching-Wen You ; Yeh-Ching Chung

Noncontiguous data access is a very common access pattern in many scientific applications. Using POSIX I/O to access many pieces of noncontiguous data segments will generate a lot of I/O requests that cause the I/O system perform poorly. Two-phase I/O, also called collective I/O, applied by MPI-IO provides a good method to optimize noncontiguous I/O operations. However, this method requires many data segments being redistributed. This redistribution needs some information been calculated in advance and lots of data segments are transmitted via network more than once. With I/O-intensive applications, the aggregate size of these data segments being redistributed becomes significant large. This additional overhead will degrade the system performance. In this thesis, we extend the collective I/O method and propose a new I/O scheme to avoid retransmission of data segments by applying RDMA gather/scatter operations supported by InfiniBand hardware. We also extend the "view" and "datatype" concepts of MPI into the file system to help complete our design. The experiments show that the method we design improves the performance and is more efficient than the collective I/O approach.

Published in:

Parallel Architectures,Algorithms and Networks, 2005. ISPAN 2005. Proceedings. 8th International Symposium on

Date of Conference:

7-9 Dec. 2005