Detection of Unknown Encrypted Malicious Traffic Based on Machine Learning

Aiming at the deficiencies of traditional encrypted malicious traffic detection methods in feature extraction accuracy, detection speed and prior knowledge dependence, this paper proposes an unknown encrypted malicious traffic detection method based on unsupervised graph learning. By constructing a dataset containing six attack types, including encrypted flood traffic, malicious web traffic and malware traffic, a traffic graph construction and preprocessing process was designed. This includes short flow aggregation, connectivity analysis and density-based pre-clustering. Combining structured feature extraction with a lightweight K-Means algorithm enables edge clustering and loss calculation. The results show that this method surpasses advanced models like HyperVision, Whisper, and DeepLog in AUC and F1 scores. It is particularly effective in detecting encrypted malicious traffic, with an improvement of about 1% over the HyperVision method.