A Performance-Oriented Comparison of Neural Network Approaches for Anomaly-based Intrusion Detection

Intrusion Detection Systems employ anomaly detection algorithms to detect malicious or unauthorized activities in real time. Anomaly detection algorithms that exploit artificial neural networks (ANN) have recently gained particular interest. These algorithms are usually evaluated and compared through effectiveness measures, which aim to quantify how well anomalies are identified based on detection capabilities. However, to the best of our knowledge, the performance characterization from the perspective of computational cost and space, training time, memory consumption, together with a quantitative analysis of the trade-offs between algorithm effectiveness and performance, have not been explored yet. In this work, we select four recently proposed unsupervised anomaly detection algorithms based on ANN, namely: REPresentations for a random nEarest Neighbor (REPEN), DevNet, OmniAnomaly, Multi-Objective Generative Adversarial Active Learning (MO-GAAL); we perform a variety of experiments to evaluate the trade-offs between the effectiveness and performance of the selected algorithms using two reference dataset: NSL-KDD and CIC-IDS-2017. Our results confirm the importance of this study, showing that none of the selected algorithms dominate the others in terms of both, effectiveness and performance. Furthermore, it shows that approaches based on Recurrent Neural Networks, which exploit the temporal dependency of the samples, have a clear advantage over the others in terms of effectiveness, while exhibiting the worst execution time.