Adaptive Defense: Zero-Day Attack Detection in NIDS With Deep Reinforcement Learning

Zero-Day attack detection in Network Intrusion Detection Systems (NIDS) refers to the ability to identify previously unseen attack patterns during testing without having been explicitly trained on those specific attacks, utilizing learned features from other known attacks. In this paper, we propose a Deep Reinforcement Learning (DRL)-based NIDS designed for Zero-Day attack detection. We use a stacked LSTM architecture to extend the learning capabilities of the DRL agent. We apply several oversampling techniques to handle the issue of class imbalance since the zero-day attack datasets are not as abundant. We use some of the most widely available benchmark datasets in NIDS domain, which all together cover a wide range of attack types, such as reconnaissance, ddoS, infiltration, injection, password attacks, brute force, dos, backdoor, and benign traffic. For example, we converted attacks to 1 and benign traffic to 0, then excluded certain attack categories (DoS and Backdoor) from the training dataset while keeping them in the test dataset. This makes those attack types zero-day attacks, as they are entirely unseen during training. We also compare which data balancing technique works better among K-means SMOTE, SMOTE, Borderline-SMOTE and ADASYN on the performance of our DRL agent. We then demonstrate how powerful our agent is by validating many datasets for remarkable success in detecting both known and unknown attacks in a zero-day manner. Our work has been made publicly available on GitHub (https://github.com/codewithkhurshed/ZDAD) to support researchers in advancing zero-day attack detection in NIDS.