Understanding Power Efficiency of TCP/IP Packet Processing over 10GbE

With the rapid evolution of network speed from 1Gbps to 10Gbps, a wide spectrum of research has been done on TCP/IP to improve its processing efficiency on general purpose processors. However, most of them did studies only from the performance perspective and ignored its power efficiency. As power has become a major concern in data centers, where servers are often interconnected with 10GbE, it becomes critical to understand power efficiency of TCP/IP packet processing over 10GbE. In this paper, we extensively examine power consumption of TCP/IP packet processing over 10GbE on Intel Nehalem platforms across a range of I/O sizes by using a power analyzer. In order to understand the power consumption, we use an external Data Acquisition System (DAQ) to obtain a breakdown of power consumption for individual hardware components such as CPU, memory and NIC etc. In addition, as integrated NIC architectures are gaining more attention in high-end servers, we also study power consumption of TCP/IP packet processing on an integrated NIC by using a Sun Niagara 2 processor with two integrated 10GbE NICs. We carefully compare the power efficiency of using an integrated NIC with using a PCI-E based discrete NIC. We make many new observations as follows: 1) Unlike 1GbE NICs, 10GbE NICs have high idle power dissipation, and TCP/IP packet processing over 10GbE consumes significant dynamic power. 2) Our power breakdown reveals that CPU is the major source of the dynamic power consumption, followed by memory. As the I/O size increases, the CPU power consumption reduces but the memory power consumption grows. Compared to CPU and memory, NIC has low dynamic power consumption. 3) Large I/O sizes are much more power efficient than small I/O sizes. 4) While integrating a 10GbE NIC slightly increases CPU power consumption, it not only reduces system idle power dissipation due to elimination of PCI-E interface in NICs, but also achieves dynamic power savings due to better processing efficiency. Our studies motivate us to design a more power efficient server architecture, which can be used in the next generation data centers.