Impact of Target Network-Specific Data on the Performance of Machine-Learning-Based Intrusion Detection System Models

Intrusion Detection Systems (IDS) using Machine Learning (ML) are deployed to enhance network security by identifying malicious activities. Their ability to generalize across different network environments remains a challenge. The problem is the performance degradation when IDS models trained on one dataset are applied to another, reducing their effectiveness in real-world deployments. This study investigates the impact of incorporating target network-specific data on IDS performance. Therefore, these are the research questions: (1) How do ML-based IDS models perform on target network data compared to their original training datasets? (2) Does integrating benign, target-specific data enhance IDS accuracy and generalization? The quantitative methodology uses CIC-IDS2017 and CSE-CIC-IDS2018 datasets to evaluate Feedforward Neural Networks (FFNN) and Convolutional Neural Networks (CNN) in cross-dataset scenarios. Performance is measured by macro F1-scores to assess the effectiveness of different training strategies. Results indicate a significant drop in detection performance on unseen network environments. While CNNs have better generalization than FFNNs, both models have challenges with cross-network adaptation. Including benign target-specific data did not lead to marked generalization improvements, highlighting the need for advanced techniques like domain adaptation and transfer learning. These findings underscore the limitations of current ML-based IDS models in diverse network settings and emphasize the necessity for improved training strategies to enhance real-world applicability.