Acceleration for MPI Derived Datatypes Using an Enhancer of Memory and Network

This paper presents a support function for MPI derived datatypes on DIMMnet-3 network interface with multi-banked extended memory, which is under development. Semi-hardwired derived datatype communication based on RDMA with hardwired gather and scatter is proposed. This mechanism and MPI using it are implemented and validated on DIMMnet-2 which is a former prototype operating on DDR DIMM slot. The performance of gather or scatter transfer of 8byte elements with large interval by using vector commands of DIMMnet-2 is 6.8 compared with software on a host. Proprietary benchmark of MPI derived datatype communication for transferring a submatrix corresponding to a narrow HALO area is executed. Observed bandwidth on DIMMnet-2 is far higher than that for similar condition with VAPI based MPI implementation on Infniband, even though poorer CPU and motherboard are used.