FSA-IDS: A Flow-based Self-Active Intrusion Detection System

As the first line of defense, an Intrusion Detection System (IDS) plays a crucial role in early cyber threat detection and successful mitigation operations. IDSs are increasingly adopting machine learning (ML)-based methods to enhance their detection engines by learning network traffic characteristics and improving the scalability. Although ML-based IDSs have proven effective in detecting diverse cyber attacks, their success is highly dependent on the input data. The scarcity of labeled benign/malicious traffic samples affects various aspects of ML model operations, e.g., benchmarking, explaining and interpreting the results, and continuous model adaptation/improvement. In addition, network traffic sample labeling is costly, complex, requires massive human effort, and is sometimes even impossible.This paper introduces a Flow-based Self-Active IDS (FSA-IDS), which is a novel framework adopting active learning (AL) into self-learning to reduce the labeling cost significantly and to realize an effective IDS. FSA-IDS improves the cyber attack detection performance while reducing false alarms. It employs a novel cluster-based sampling approach that facilitates the labeling automation process and minimizes expert involvement by up to 47% compared to various baselines. We evaluate FSA-IDS using two real-world network traffic datasets comprising a wide range of benign and malicious network traffic samples.