P4RSS: Load-Aware Intra-Server Load Balancing with Programmable Switching ASICs

Off-the-shelf x86 servers are widely deployed as middleboxes in edge and public clouds, such as cloud gateways and load balancers. They follow the “run-to-completion” model and achieve parallel traffic processing by distributing packet flows across multiple CPU cores using the RSS (receive side scaling) capability of NICs. However, RSS can cause inter-core load imbalance as it conducts stateless hashing without considering the CPU core utilization. As a result, multiple heavy-hitter flows can potentially overload a single CPU core when they are hashed onto that core. In this research, we propose P4RSS, a load-aware intra-server load balancing solution that leverages the P4 data plane. Specifically, a P4 ASIC is placed in front of the CPU to perform stateful traffic load balancing among multiple CPU cores based on real-time monitoring of core utilization. In addition, flow affinity maintenance and heavy hitter throttling are also offloaded to the P4 ASIC to free up valuable CPU computing resources. P4RSS can be implemented in the form of either hyper-converged server switches or P4-based SmartNICs. Evaluation results demonstrate that P4RSS reduces the standard deviation of CPU core utilization by 22%~53% compared to RSS. This not only improves the stability of middleboxes but also allows for higher CPU utilization without overprovisioning.