SGMFuzz: State Guided Mutation Protocol Fuzzing

Protocol implementations are fundamental components in network communication systems, and their security is crucial to the overall system. Fuzzing is one of the most popular techniques for detecting vulnerabilities and has been widely applied to the security evaluation of protocol implementations. However, due to the lack of machine-understandable prior knowledge and effective state-guided strategies, existing protocol fuzzing tools tailored for stateful network protocol implementations often suffer from shallow state coverage and generate numerous invalid test cases, thereby impacting the effectiveness of the testing process. In this letter, we introduce SGMFuzz, a grey-box fuzzing tool that combines a state-guided mutation mechanism to detect security vulnerabilities in protocol implementations. SGMFuzz uses the feedback collected during fuzzing to construct a finite-state machine, which aids in a deeper exploration of the program. Additionally, we design a message-aware module to enhance the tool’s ability to generate valid test cases. Our evaluation demonstrates that, compared to the most advanced and widely used network protocol fuzzing tools, SGMFuzz increases the number of discovered execution paths by over 15% on average and improves state transition coverage by over 10%, providing a more comprehensive security assessment of protocol implementations.