SEMPLAR: high-performance remote parallel I/O over SRB

One of the challenges in high-performance computing is to provide users with reliable, remote data access in a distributed, heterogeneous environment. The increasing popularity of high-speed wide area networks and centralized data repositories lead to the possibility of direct high-speed access to remote data sets from within a parallel application. In this paper, we describe SEMPLAR, a library for remote, parallel I/O that combines the standard programming interface of MPI-10 with the remote storage functionality of the SDSC storage resource broker (SRB). SEMPLAR relies on parallel TCP streams to maximize the remote data throughput in a design that preserves the parallelism of the access all the way from the storage to the application. We have provided I/O performance results for a high-performance computing workload on three different clusters. On the NCSA TeraGrid cluster, the ROMIO perf benchmark attained an aggregate read bandwidth of 291 Mbps with 18 processors. The NAS btio benchmark achieved an aggregate write bandwidth of 74 Mbps with 16 processors. The benchmark results are encouraging and show that SEMPLAR provides applications with scalable, high-bandwidth I/O across wide area networks.