10-Gigabit iWARP Ethernet: Comparative Performance Analysis with InfiniBand and Myrinet-10G

iWARP is a set of standards enabling remote direct memory access (RDMA) over Ethernet. iWARP supporting RDMA and OS bypass, coupled with TCP/IP offload engines, can fully eliminate the host CPU involvement in an Ethernet environment. With the iWARP standard and the introduction of 10-Gigabit Ethernet, there is now an alternative path to the proprietary interconnects for high-performance computing, while maintaining compatibility with existing Ethernet infrastructure and protocols. Recently, NetEffect Inc. has introduced an iWARP-enabled 10-Gigabit Ethernet channel adapter. In this paper we assess the potential of such an interconnect for high-performance computing by comparing its performance with two leading cluster interconnects, infiniband and myrinet-10G. The results show that the NetEffect iWARP implementation achieves an unprecedented latency for Ethernet, and saturates 87% of the available bandwidth. It also scales better with multiple connections. At the MPI level, iWARP performs better than infiniband in queue usage and buffer re-use.