Machine Learning Techniques For Design Of Intrusion Detection System For Big Data Networks

As digital technology advances, gigabytes and terabytes of data are now generated every second. Businesses in a variety of industries are finding that using the internet to manage their resources and transactions is useful. Given the value of data and the need to safeguard its security and privacy, securing big data remains a major challenge for all solutions. Due to the exponential expansion of network data, intrusion detection is becoming increasingly important, and manual analysis would be either impossible or take the same amount of time as analyzing it. As a result, there is an urgent need for an automated system capable of extracting relevant information from enormous amounts of hitherto untapped data when it comes to network intrusion detection. Data mining can perform a variety of tasks, such as clustering, prediction, classification, and the extraction of association rules between data pieces. In this research, we explore the application of machine learning methodologies to the task of creating intrusion detection systems for large data networks. As an input for this methodology, the NSL KDD data set is used. In the first step of this process, useful features from the NSL KDD data set are extracted using the CFS-correlation technique to feature selection. There are 41 features included in the NSL KDD data collection. Following the use of the CFS method, the total number of features was cut down to 16. Following this, machine learning methods are used to the NSL KDD data set in order to features the malware data and make predictions based on the data using the 16 characteristics.