Network traffic verification based on a public dataset for IDS systems and machine learning classification algorithms

One method of monitoring network traffic is to detect anomalies where the primary goal is to distinguish normal from abnormal network traffic. In this research we used classification algorithm with best result with public IDS (Intrusion Detection System) dataset on real network dataset. NetFlow data collected from network devices are a good source of input for machine learning classification algorithms that can identify anomalies with high certainty based on known data flows. In this paper, several machine learning classification algorithms (Random Forest, K-Nearest Neighbors, Naive Bayes and Support Vector Machines) were used, for recognizing nine types of network attacks (traffic anomalies) classified in UNSW-NB15 dataset. The collected NetFlow data of real traffic on network devices and simulated network attacks were tested on a proposed machine learning model to determine the success of detecting security incidents. F2 and AUC (Area under the ROC curve) scores were used to measure the performance of classification algorithms, as UNSW-NB15 is a highly unbalanced learning dataset. In both classification of reference dataset and dataset of real traffic the best result achieved Random Forest algorithm with AUC sore of 97% and 95%.