Unsupervised Scanning Behavior Detection Based on Distribution of Network Traffic Features Using Robust Autoencoders

In intrusion detection systems, the application of machine learning to detection methods has attracted interest from vendors and researchers, because most existing signature-based methods cannot cope with increasing incidents, and some machine learning methods, particularly neural networks (NNs), have achieved remarkable results in computer vision and other fields. In applying NNs to intrusion detection based on network traffic, it is very important to adopt unsupervised methods instead of supervised ones, because unsupervised methods are better for detecting unseen attacks, and network traffic behavior depends very much on each host or network, so that it is not realistic to prepare labels for each environment. In previous studies, however, the application of unsupervised methods has not been investigated much. A flow-based detection method using replicator NNs, or autoencoders, is one of a few unsupervised methods. The experimental results showed that unsupervised anomaly detection using autoencoders is certainly effective for detecting scanning behavior, however, the many false positives are still challenging. In this paper, flow-based intrusion detection using robust autoencoders (RAEs) is proposed. RAEs split the training data into benign and outlier features in an unsupervised manner, and the model learns low-dimensional representations only from the benign data, and splitting and learning are repeated alternately. We argue that RAEs reduce false positives in the test data by removing outliers in the training data. In this paper, a real-world traffic dataset, the MAWI (Measurement and Analysis on the WIDE Internet), was used for evaluations. Experiments showed that false positives were significantly reduced and attacks were detected more easily.