ADSIM: Network Anomaly Detection via Similarity-aware Heterogeneous Ensemble Learning

The last decade has seen increasing application of machine learning to various tasks, including network anomaly detection. But anomaly detection methods using a single machine learning algorithm often fail to perform well, since network traffic can have complex and changeable patterns. Therefore, many solutions based on ensemble learning have been proposed to address this problem. However, previous studies have a essential drawback that they overlook the similarity between the weak classifiers, which may degrade the detection ability of the model. What's more, prior work use offline and supervised algorithms, which means a large amount of memory and reliable labels are necessary during the training period. In this paper, we propose ADSIM, an online, unsupervised, and similarity-aware network anomaly detection algorithm based on ensemble learning. In the training phase, ADSIM incrementally maintains a distance matrix to record the similarity between the classifiers and uses hierarchy clustering to group similar classifiers. In the detecting phase, each cluster will be assigned a weight based on the consistency of the classifier outputs within it. We evaluate ADSIM on two datasets, MAWILab and CIC-IDS-2017, and the results show that ADSIM can accurately detect various anomalies and outperforms state-of-the-art ensemble learning methods.