Using of Transformer-Based Language Models to Separate Traffic Packets of Different Protocols

Recognition of network protocols is one of the most important steps in the intelligent algorithms for detecting incidents and anomalies in information security systems. In cases where there is no formal specification and documentation of protocols, it is necessary to apply approaches from the field of reverse engineering of protocols. The accuracy of some of these approaches can be improved by applying them not to the entire raw traffic dump, but to separate groups corresponding to one protocol stack.In this paper, a method is proposed for dividing the traffic dump into groups in accordance with the packet protocol stack. A key feature of the approach is the use of transformer-based models to construct contextualized vector representations of traffic packets, which will then be used as features in clustering. We demonstrate through a series of experiments that the proposed approach efficiently works on different types of network traffic and allows us to improve the quality of packets clustering, surpassing the previously proposed methods.