A Deep Learning-Based Hybrid Model for Optimal Anomaly Detection

Intrusion Detection Systems (IDS) play a critical role in ensuring the security of an organization's network infrastructure by detecting potential threats. One approach used by many IDS tools is to analyze system calls using flow-based techniques. As the security field continues to evolve, IDS-specific research has become a primary focus, with numerous studies and methods proposed to improve detection accuracy. This study proposes a new hybrid approach for malware detection using deep learning models, specifically the Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN). Additionally, we incorporate an Autoencoder (AE), a type of neural network used for unsupervised learning and data compression. To evaluate the performance of our proposed models, we used the open-source benchmark dataset ADFA-LD for training and assessment. Our experimental analysis demonstrates that our proposed hybrid model can achieve promising results in detecting malware with high accuracy. An autoencoder allows our models to learn and identify patterns that may indicate malware, even without labeled training data. By combining the strengths of multiple deep learning models, we achieved better detection accuracy than traditional flow-based IDS methods.