Enhancing Detection of Malicious Traffic Through FPGA-Based Frequency Transformation and Machine Learning

In recent years, with the development of modern network, in order to avoid threats caused by cyber attacks, it is important to understand how to implement effective security measures. Malicious traffic detection is an advanced technique, it employs several approaches to distinguish traffic whether it is benign or malicious. Traditional malicious traffic detection methods are usually based on pre-defined signatures. However, limited by the size and timeliness of the signature library, they are usually unable to detect unknown cyber attacks such as zero-day attacks or new malware variants. In order to solve this problem, we propose a machine learning based method to detect realtime malicious traffic. It is divided into two parts: feature extraction on FPGA and abnormal traffic detection on Linux host machine. On the feature extraction part, instead of using conventional network traffic features, we propose a frequency transformation based feature extraction method to extract frequency domain features from network traffic. At the same time, in order to improve the speed of feature extraction and reduce CPU resource usage, we implement the processes required for feature extraction inside the FPGA board. On the abnormal traffic detection part, we use AF-Packet and ring buffer to capture the features, and load a pre-trained model into the CatBoost framework in advance to execute inference process. We evaluate our proposed system on a Xilinx Alveo U50 accelerator card and a Linux host machine. The evaluation results show that we achieve about 0.98 detection accuracy with low resource usage and good realtime detection throughput.