Automated Protocol Reverse Engineering with Message Format Inference

Although protocol specifications provide a way to interpret messages that were communicated, such information may not be readily available for proprietary protocols and often differs between original equipment manufacturers. Protocol reverse engineering (PRE) tries to make sense of messages transmitted over networks or systems to understand the inner workings of protocols and their messages. This work proposes a 3 staged PRE approach that observes bitflips in time-series messages to identify patterns in deterministic data. It monitors changes in behaviour of binary protocols and thereby infers the structure of message formats. First, the pre-processing stage groups messages by their type and computes bitflips. The field type inference stage then tries to determine the field types for individual bits. Finally, the post inference validation stage establishes the field boundaries of the inferred message formats. The approach was evaluated on publicly available Controller Area Network (CAN) dataset against automated CAN payload reverse engineering and reverse engineering of controller area networks methods. Experiments showed the proposed approach infers finer message formats details. Keywords-protocol reverse engineering, message format inference, binary protocol, Controller Area Network, bitflip