On optimizing I/O through InfiniBand RDMA for commodity clusters
Allan, B.
Chen, H.
Cranford, S.
Minnich, R.
Rudish, D.
Ward, L.
Sandia Nat. Labs., Livermore, CA, USA
This paper appears in: Cluster Computing and Workshops, 2009. CLUSTER '09. IEEE International Conference on Publication Date: Aug. 31 2009-Sept. 4 2009
On page(s):
1
- 6
Location: New Orleans, LA
ISSN: 1552-5244
ISBN: 978-1-4244-5011-4
Digital Object Identifier: 10.1109/CLUSTR.2009.5289146
Current Version Published: 2009-10-16
Abstract
Our goal was to enable pNFS as a high performance parallel file system by using network file system (NFS) storage objects and InfiniBand remote direct memory access (RDMA) transport in the Linux mainline. One obstacle to this approach was the performance bottleneck in NFS/RDMA streaming-writes from the compute nodes. We benchmarked, tuned, and improved the streaming-write efficiency of the Linux NFS client. However, deeper analyses of the benchmarks and the various I/O short circuit schemes established upper bounds on the performance of the NFS client - even with an infinitely fast network - so that the performance was substantially less than the theoretical streaming bandwidth of the fast interconnections. The complex interactions between the Linux virtual file system, Linux virtual memory management, and the IB network subsystems apparently impose a limit on further improvement.
Index
Terms
Available to subscribers and IEEE members.
References
Available to subscribers and IEEE members.
Citing Documents
Available to subscribers and IEEE members.
You are not
logged in.
Guests
may access Abstract records free of charge.
Cart
