PM/InfiniBand-FJ: a high performance communication facility using InfiniBand for large scale PC clusters

This work describes a design of high performance communication facility called the PM/InfiniBand-FJ using InfiniBand interconnect for large scale PC clusters. The PM/InfiniBand-FJ has developed to realize higher application performance than commercial supercomputers and comparable availability to them. Since the specification of InfiniBand interconnect is designed for communication among servers and I/Os, there are some issues to use InfiniBand for high performance computation on over 1000 node PC clusters. Therefore, the PM/InfiniBand-FJ solves the issues by expanding the original specification of InfiniBand. We have implemented the PM/InfiniBand-FJ on SCore cluster system software, and evaluated the communication and application performance. The performance results show that a 913.2 MB/s of bandwidth and 15.6 /spl mu/s round trip time have been achieved on Xeon 2.8GHz PC with ServerWorks GC LE chipset. The result of NAS parallel benchmark shows that the 128 node result of IS Class B on PM/InfiniBand-FJ is 1.52 times faster than that of PM/MyrinetXP using Fujitsu PR1MERGY RX200 PC cluster (Xeon 3.06GHz).