Packet Field Tree: a hybrid approach to automated protocol reverse-engineering

Information and communications technology (ICT) systems exchange data through agreed network protocols. These constrain how ICT systems send network packets through a protocol specification. Protocol specifications are useful knowledge for building intrusion detection systems to monitor system behaviour, or for performing penetration testing to find system vulnerabilities. However, protocol specifications are typically verbose documents, and when new protocols are tested, considerable manual effort is required to translate these specifications into digital models of expected protocol behaviour. Automated protocol reverse-engineering (APRE) is a precursor to generating such digital models for the aforementioned applications. In this paper, we introduce the Packet Field Tree (PFT), the first hybrid approach combining the advantages of heuristic and supervised-learning methods, and apply it to one aspect of the APRE problem concerning protocol formats. Our model infers the boundaries between fields of an unknown protocol with a superior mean perfection, an APRE-specific quasi-accuracy measure, of 0.27 over a variety of 21 protocols. This is an improvement of at least 0.11 compared to three existing APRE models. Although this is far from a perfect score of 1.00, highlighting the difficulty of APRE, our PFT also infers the field syntax types, which is a feature unique to our model. To measure this feature, we also propose the PFT-Score, a starting point for holistic evaluation of APRE outputs.