Assessing Single-Message and Multi-Node Communication Performance of InfiniBand

We present a micro benchmark suite to evaluate InfiniBandTM implementations with regards to single message performance and the addressing of many hosts. We use a 1:n communication pattern to assess the latency and bandwidth for all different combinations of InfiniBandsTM transport services and functions. The results gathered in this study are used to optimize MPI collective communication operations where 1:n communication schemes are not used widely today. We show that applications as well as collective algorithms can benefit from sending multiple messages in a single round. Moreover, the results will be used to choose the transport service and function to develop InfiniBandTM optimized collective communication functions. Our study compares all available transport options and shows that single packet sends can be very expensive compared to multi packet sends.