Securing Big Data Ecosystem with NSGA-II and Gradient Boosted Trees Based NIDS Using Spark

Big Data represents the massive data that is capable of saving billions of dollars in medical industry, sports, business, military, black box operations and so forth, just by visualizing and analyzing data within no time. In this paper, we propound a Network Intrusion Detection System (NIDS) model based on Gradient Boosted Trees classifier for network traffic analysis in order to distinguish between legitimate and malicious packets that utilize Nondominated Sorting Algorithm (NSGA-II) for feature selection procedure. We use Apache Spark which is built on Hadoop, to deal with the computational complexities of Big Data on NetBeans platform. Moreover, we utilize NSL KDD cup 99 dataset to perform the experimental analysis.