Network traffic anomaly detection based on growing hierarchical SOM

Network anomaly detection aims to detect patterns in a given network traffic data that do not conform to an established normal behavior. Distinguishing different anomaly patterns from large amount of data can be a challenge, let alone visualizing them in a comparative perspective. Recently, the unsupervised learning method such as the K-means [3], self-organizing map (SOM) [2], and growing hierarchical self-organizing map (GHSOM) [1] have been shown to be able to facilitate network anomaly detection [4][5]. However, there is no study addressing both mining and detecting task. This study leverages the advantage of GHSOM to analyze the network traffic data and visualize the distribution of attack patterns with hierarchical relationship. In the mining stage, the geometric distances between each pattern and its descriptive information are revealed in the topological space. The density and the sample size of each node can help to detect anomalous network traffic. In the detecting stage, this study extends the traditional GHSOM and uses the support vector machine (SVM) [6] to classify network traffic data into the predefined categories. The proposed approach achieves (1) help understand the behaviors of anomalous network traffic data (2) provide effective classification rule to facilitate network anomaly detection and (3) accumulate network anomaly detection knowledge for both mining and detecting purpose. The public dataset and the private dataset are used to evaluate the proposed approach. The expected result is to confirm that the proposed approach can help understand network traffic data, and the detecting mechanism is effective for identifying anomalous behavior.