Evaluating Deep Learning Based Network Intrusion Detection System in Adversarial Environment

Deep learning plays a vital role in network security field. Furthermore, the performance of deep learning based Network Intrusion Detection System (NIDS) is satisfactory and even better than the traditional methods. However, recent researches show that the accuracy of deep learning based image classification declines sharply when facing the adversaries. Regarding NIDS, there is no comprehensive evaluation of whether it will also be affected by attack models. In this paper, we propose a framework of Evaluationg Network Intrusion Detection System (ENIDS) to research the robustness of deep learning based NIDS in adversarial environment. More specifically, we train four target models (e.g., Deep Neural Networks, Support Vector Machine, Random Forest and Logistic Regression) over the benchmark dataset NSL-KDD. Furthermore, four advanced attack models (e.g., Projected Gradient Descent attack, Momentum Iterative Fast Gradient Sign Method, L-BFGS attack and SPSA attack) are used to generate adversarial samples. Finally, comprehensive and extensive experiment results show that the effectiveness of deep learning based NIDS is greatly undermined by the adversaries.