Characterization of network processing overheads in Xen

I/O virtualization techniques developed recently have led to significant changes in network processing. These techniques require network packets go through additional layers of processing. These additional layers have introduced significant overheads. So it is important to understand performance implications of this additional processing on network processing (TCP/IP). Our goals in this paper are to measure network I/O performance in a Xen virtualized environment and to provide a detailed architectural characterization of network processing highlighting major sources of overheads and their impact. In this paper, we study two modes of I/O virtualizations: 1) running I/O service VM along with the guest on the same CPU, and 2) running I/O service VM on a separate CPU. We measure TCP/IP processing performance in these two modes and compare it to that of on the native Linux machine. Our measurements show that both Rx and Tx performance suffer by more than 50% in virtualized environment. We have noticed that pathlength has increased by 3 to 4 times than that of the native processing. Most of this overhead comes from the Xen VMM layer and Dom0 VM processing. Our data also shows that running the Dom0 VM on a separate CPU is more expensive than running both Dom0 and guest VM on the same CPU. We provide a detailed characterization of this additional processing which we hope will help the Xen community focus on right areas for optimization.