Unsupervised Anomalous Traffic Detection Based on Cascading Representation and Multiple-Clustering

To address the challenge of lacking labeling information in network traffic anomaly detection, we propose an unsupervised anomaly detection algorithm based on cascading representation and multiple-clustering. First, this paper designs a cascading representation method to obtain discriminative traffic features. We create a residual auto-encoder that employs generative learning to capture generic statistical expressions. We augment the time-series features in various ways and leverage contrastive learning to capture discriminative representations. We develop a host identification task based on triplet loss, enhancing feature discrimination in addition to cascading two feature representations. Second, we design a multiple-clustering-based algorithm for anomalous traffic detection. By performing KMeans on the cascading feature representations, we obtained clustering centers to replace each cluster and avoid the problem of unbalanced abnormal traffic. By applying DBSACAN clustering with a tree structure to these clustering centers and using the height of the anomalies to calculate the traffic anomaly score, we can detect abnormal network traffic flexibly. Finally, we conducted experimental validation on several commonly used intrusion detection datasets. The experimental results show that the method proposed in this paper generally outperforms the comparison methods.