A Co-Optimization of Software and Hardware for PCIe-Based Small Packet DMA Transfer

To meet the development needs of high-performance networks, the efficiency of PCIe DMA with small packet transmissions emerges as a critical performance bottleneck. For the purpose improving DMA transfer for small packets over PCIe, we focus on the symbiotic co-optimization of software and hardware. By enhancing the software driver with descriptor prefetching, harnessing the outstanding capability at the hardware level, and using multicore for parallel processing, the system’s bandwidth has been significantly improved, particularly for the small packets transfers. Postoptimization, we test the DMA read and write bandwidth and utilization rate performance on VC709 FPGA. In a randomized test scenario, the average DMA read bandwidth is approximately 5.3 GB/s, representing a 123% improvement compared to the unoptimized system bandwidth. The average DMA write bandwidth is approximately 5.8 GB/s, representing a 136% improvement compared to the unoptimized system bandwidth. Additionally, in the randomized test scenario, the average read and write latencies improved by 15.12% and 23.96%, respectively.

I. Introduction

With the development of high-performance networks, the demand for high throughput, low-latency networks is increasing. According to the analysis of data center traffic characteristics by the Benson et al. [1], 60% of data flows do not exceed 1 KB in size, and these small data flows are sensitive to latency. Therefore, it is necessary to optimize the bandwidth and latency of small packet transmission in network communication.