Elephant flow detection in datacenters using OpenFlow-based Hierarchical Statistics Pulling

This paper proposes an effective elephant flow detection in datacenters. Equal Cost MultiPath (ECMP) is a popular routing mechanism to achieve load balancing in datacenter networks. However, ECMP only considers equally distributing the flows over equal-length paths, but does not take the size of the flows into account. Previous studies show that detecting and rerouting elephant flows (flows that transfer significant amount of data) effectively can lead to a 113% improvement in aggregate throughput compared with the simple use of ECMP. A naive way to detect the elephant flows is to pull statistics from each flow independently. Since "the elephant and mouse phenomenon" suggests that there are only very few elephant flows in a datacenter network, it's not efficient to collect information from each flow. Therefore, we propose a Hierarchical Statistics Pulling (HSP) mechanism using OpenFlow protocol to save bandwidth consumption and processing time. In order to further improve the performance of HSP, two supplement functions called elephant store and range splitting are developed. Our approaches can detect elephant flows with only a small amount of network bandwidth consumption. We use Mininet emulator and mathematical analysis to verify our methods. Both of them confirm the benefits of our approaches.