Achieving Optimal Edge-based Congestion-aware Load Balancing in Data Center Networks

Load balancing is the key to improve the performance of data center networks (DCN). The de facto scheme, Equal Cost MultiPath (ECMP) is well-known for its inferior performance due to the coarse-grained and congestion-oblivious nature. As such, more fine-grained and congestion-aware schemes recently emerge. However, current network-based schemes require special modifications to existing switch hardware that makes them hard to deploy. In addition, current edge-based schemes can not achieve optimal load balancing performance due to the lack of accurate in-network congestion information. We propose EMAN, as the first step to achieve optimal load balancing in DCN at the edge. Instead of trying to get the exact network congestion condition (hard to be done at the edge), EMAN directly distributes outbound traffic from a sending endhost proportional to bandwidth of each path, to balance the path utilization. By dynamically updating each path's available bandwidth using feedback from the receiver, EMAN can gracefully react to network asymmetries caused by failure and flow competition. Both testbed and simulation results show that EMAN can improve the performance by up to 80% (testbed) compared to ECMP, and 66% (simulation) compared to the latest edge-based congestion-aware schemes. EMAN has been implemented as a hot-pluggable Linux kernel module which is transparent to existing applications and kernel TCP stack.