Data Processing and Model Selection for Machine Learning-based Network Intrusion Detection

Signature-based Intrusion Detection Systems (IDSes) such as Snort, BRO or Suricata depend on specific patterns and byte sequences in network traffic to detect intrusions; hence, they cannot prevent intrusions for unknown zero-day attacks. Various anomaly-based IDSes that have been proposed based on machine learning (ML) techniques incur high false positives. To overcome this, we explore different types of data processing, i.e. data balancing, feature correlation, normalization, and feature reduction, and whether they are necessary for datasets with different feature dimensions: Coburg Intrusion Detection Data Sets (CIDDS) with five features and Knowledge Discovery and Data Mining (KDD) with 41 features. Further, we perform model selection by comparing the performance of various linear and non-linear classifiers. Generally, our results show that nonlinear classifiers outperformed linear ones and that using data balancing and normalization improves the overall accuracy for most classifiers.