Feature Selection Using Pearson Correlation with Lasso Regression for Intrusion Detection System

The growth of Internet users and traffic drives significant changes in the network security domain. Computer networks become increasingly vulnerable to attack by irresponsible parties, which can potentially cause substantial loss and damage due to information stealing, sending malicious packets, and overtaking network resources. While the effort to secure the network has been conducted persistently, unfortunately, the variety and volume of cyber threats have continuously increased. As such, there is a demand for an effective and efficient attack detection model to prevent this catastrophic network failure by implementing feature selection techniques to reduce the dimension of the feature on a dataset. We proposed a feature selection method combining the Pearson Correlation method with Lasso Regression to address that need. The Pearson Correlation is used to select the best feature based on its degree of relationship. After that, the selected result is optimized using Lasso Regression to achieve the best features for the IDS model. This study is conducted using the UNSWNB-15 dataset and it is revealed that this proposed method significantly improves the SVM classifier's accuracy and false positive rate from 76.45% to 97.17% and 20.67% to 1.44%, respectively.