Scalable Design and Implementations for MPI Parallel Overlapping I/O
Liao, W.-K.
Coloma, K.
Choudhary, A.
Ward, L.
Russell, E.
Pundit, N.
Electr. Eng. & Comput. Sci. Dept., Northwestern Univ., Evanston, IL;
This paper appears in: Parallel and Distributed Systems, IEEE Transactions on
Publication Date: Nov. 2006
Volume: 17,
Issue: 11
On page(s): 1264-1276
ISSN: 1045-9219
INSPEC Accession Number: 9197264
Digital Object Identifier: 10.1109/TPDS.2006.163
Current Version Published: 2006-10-02
Abstract
We investigate the message passing interface input/output (MPI I/O) implementation issues for two overlapping access patterns: the overlaps among processes within a single I/O operation and the overlaps across a sequence of I/O operations. The former case considers whether I/O atomicity can be obtained in the overlapping regions. The latter focuses on the file consistency problem on parallel machines with client-side file caching enabled. Traditional solutions for both overlapping I/O problems use whole file or byte-range file locking to ensure exclusive access to the overlapping regions and bypass the file system cache. Unfortunately, not only can file locking serialize I/O, but it can also increase the aggregate communication overhead between clients and I/O servers. For atomicity, we first differentiate MPI's requirements from the portable operating system interface (POSIX) standard and propose two scalable approaches, graph coloring and process-rank ordering, which can resolve access conflicts and maintain I/O parallelism. For solving the file consistency problem across multiple I/O operations, we propose a method called persistent file domains, which tackles cache coherency with additional information and coordination to guarantee safe cache access without using file locks
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
Cart
