nSwitching: Virtual Machine aware relay hardware switching to improve intra-NIC Virtual Machine traffic

Recent development on Ethernet switching to provide Single Root I/O Virtualization (SR-IOV) on network interface cards (NICs) improves Ethernet throughput for Virtual Machines (VMs) and lowers CPU loads. SR-IOV creates multiple receive queues on a NIC, directly accessible by VMs for frames coming from sources external to the Ethernet port. This virtualization of Ethernet ports and the presentation of frames directly to VMs eliminates a major cause for CPU loading by reducing the interrupts for receipt of inbound frames. However, SR-IOV cannot provide switching support for two VMs on the same computer; the only existing switching option is software-based switching in the hypervisor, which limits throughput and results in high CPU utilization. New industry standards 802.1Qbg and 802.1Qbh assist Ethernet traffic between VMs, but they require costly replacement of both Ethernet NICs and the data center external physical switch infrastructure. In this paper, we propose a new design by integrating a new Ethernet switching functionality into the NIC, which is called nSwitch, to enable hardware-based switching for inter-VM traffic on a single computer that has a single or multi-socket, multi-core CPU. Compared with software-based switching in the hypervisor, this enhancement greatly reduces CPU utilization and permits efficient traffic monitoring for on-board inter-VM I/O. Furthermore, it eliminates the back-and-forth usage of external port or channel bandwidth for internal VM communications.

I. Introduction

Traditional data center network switching architecture involves the definition of switching platforms, port bandwidth, physical medium connectivity, virtual local-area network (VLAN) and Internet Protocol (IP) addressing, fail-over mechanisms, port bonding, quality of service (QoS) and security. With the introduction of improved switching protocols and virtualization, increased utilization of hardware has imposed many design challenges. Trill [5], a recent replacement for spanning tree, eliminates unused redundant ports across data center switches and permits continuous up time during network technology refresh or maintenance. A more dramatic change is the virtualization of Operating Systems (OS) which pushes CPU and I/O utilization to levels unachievable without Virtual Machines (VMs). Hypervisors and VM clients are improving hardware reduction, beyond that achieved by stacking of multiple applications on a single computer.