IBRMP: A Reliable Multicast Protocol for InfiniBand

Modern distributed applications in high-performance computing (HPC) fields often need to disseminate data efficiently from one cluster to an arbitrary number of others by using multicast techniques. InfiniBand, with its high-throughput, low latency and low overhead communications, has been increasingly adopted as an HPC cluster interconnection. Although Infini Band hardware multicast is efficient and scalable, it is based on Unreliable Data grams (UD) which cannot guarantee reliable data distribution. This makes Infini Band multicast not the best fit for modern distributed applications. This paper presents the design and implementation of a reliable multicast protocol for Infini Band (IBRMP). IBRMP is based on Infini Band unreliable hardware multicast, and utilizes Infini Band Reliable Connection (RC) to guarantee data delivery. According to our experiments, IBRMP takes full advantage of Infini Band multicast which reduces communication traffic significantly. In our testing environment, using IBRMP is up to five times faster than using only RC to disseminate data among a group of receivers. Compared to the MPIBcast, IBRMP is able to provide an equivalent low latency service in addition to its efficiency in large amount of data transmission.