Towards Effective and Efficient Error Handling Code Fuzzing Based on Software Fault Injection

Software systems often encounter various errors or exceptions in practice, and thus proper error handling code is essential to ensure the reliability of software systems. Unfortunately, error handling code is often bug-prone, while sufficiently testing them is challenging as such code often cannot be triggered under normal conditions. Motivated by this, recent studies have proposed to leverage software fault injection (SFI) based fuzzing to discover potential bugs in complicated error handling code. Despite the promising results achieved, their effectiveness and efficiency are still compromised in practice due to the huge search space of error sites, inadequate fuzzing guidance, and the overhead induced by context-sensitive SFI. To achieve effective and efficient testing of error handling code, this study presents AFL-FI, which first utilizes a similarity-based method to identify suspicious error sites, and then incorporates the idea of error site coverage to guide the fuzzing process. Finally, the design of lightweight context-sensitive SFI enables AFL-FI to execute test cases efficiently. We evaluate AFL-FI on eight large-scale open-source projects, and the results show that it can outperform existing state-of-the-art fuzzing tools significantly in terms of branch code coverage. More importantly, AFL-FI has discovered 13 previously unknown bugs, and all of them have been confirmed while 12 of them have been fixed. Besides, our evaluation also demonstrates that all the key designs of AFL- F I are effective that contribute significantly to its overall performance.