Network Traffic Measurement Algorithm Based on Sampling for Big Network Data

Network traffic measurement is significant for network security and network management. As network bandwidth increases and internet applications varies, network big data is bringing new challenge for network traffic measurement. Because the existing network traffic measurement mainly processes network traffic data by the centralized method, it is very difficult to meet the application needs of massive data. According to scalability of network traffic measurement and load imbalance, the network traffic measurement based on MapReduce is researched. Elephant flow identification is an important application field in network traffic measurement, so the elephant flow identification algorithm based on sampling is proposed. Hadoop cluster test environment is built and the real network traffic is used to validate performance of the elephant flow identification algorithm on Hadoop cluster. The experimental results illustrate that the proposed algorithm has good scalability and load balancing.