Few-shot Encrypted Malicious Traffic Classification via Hierarchical Semantics and Adaptive Prototype Learning

While encrypted traffic improves security, it is also used by attackers to hide the transmission content to evade detection. Currently, traffic side-channel features combined with Deep Learning (DL) are widely used for malicious traffic classification, but traditional DL-based methods require large training samples and struggle with new threats. Prototypical networks in meta-learning have been effective in few-shot malicious traffic classification. However, existing methods face challenges such as "overlooking hierarchical traffic dependencies" and "bias in class prototype generation". The former means that the existing methods lack the representation design based on the hierarchical structure of traffic, resulting in insufficient feature extraction, and the latter indicates that the existing methods struggle to capture the diverse distribution of traffic features, resulting in unstable classification performance. To address the above problems, this paper proposes a few-shot encrypted malicious traffic classification method based on Hierarchical Semantics and Adaptive Prototype Learning Network (HANet). First, network traffic’s fine-grained features are represented in a multi-level matrix, with a hierarchical network structure designed to extract features comprehensively. Then, class prototypes are dynamically generated using a neighborhood partitioning method to balance simple and complex traffic feature distributions, enhancing generalization. Experiments on the CICandMal2017 dataset show that HANet offers significant performance over other few-shot malicious traffic classification methods. HANet has achieved a classification accuracy of more than 80% with only 5, 10, and 15 labeled traffic samples, realizing effective detection of few-shot encrypted malicious traffic.