Packet-based load-balancing in fat-tree based data center networks

In a data center with TCP/IP communications, it is generally believed that packet-based load balancing is not suitable because the associated packet out-of-order problem will significantly lower the network utilization. In this paper, we first show that if packet-based load balancing is performed properly in a fat-tree based data center, the packet out-of-order problem is not as severe as most researchers believed. This is due to the fact that multiple (minimal) paths between any given pair of servers in a fat-tree are of the same hop count. If packets are evenly routed onto different paths, they will experience similar delay performance. As a result, the packet out-of-order arrivals at the receiver are usually within a small sequence number range. Notably, the fast retransmit (FR) algorithm in TCP will be triggered for resending the “lost” packet if three duplicate ACKs are received (i.e. FR threshold is three). To provide leeway for out-of-order packet arrivals due to packet-based load balancing, we propose to judiciously increase the FR threshold. Simulation results show that FR threshold values between 6 and 9 can effectively suppress unnecessary fast retransmits and at the same time, the impact to real packet losses is minimal. Compared to a flow-based load balancing scheme, we found that our packet-based load balancing with modified TCP consistently provides higher goodput and noticeably smaller delay.