A parallel file transfer protocol for clusters and grid systems

Parallel file transfer protocol (PFTP) introduces the concept of data transfer using multiple parallel data paths between clusters. It makes use of striped data distributed across the multiple IO-nodes of a parallel file system on clusters. The protocol uses the striping format of the parallel file system to push multiple stripes of the file over multiple TCP connections. The flexibility in PFTP allows user to use different file transfer applications over these TCP connections. In this paper we describe the design architecture and implementation of PFTP. Several experiments explain how it improves the data transfer rate. We also provide performance results of PFTP for a varying file data size of 100 MB to 4GB. The flexible, secure, fault tolerant and high performance design of PFTP can be useful for large data grids.