Towards Better Static Analysis Bug Reports in the Clang Static Analyzer

Static analysis is a method increasingly used for finding bugs and other deviations of software systems. While it fits well for the modern development environment and is capable to catch issues earlier than the manual code review or the various testing methodologies, human experts are still deeply involved in the evaluation of whether the tool reported a true finding or a false positive. This creates a serious bottleneck in the development flow. While many efforts have been made to improve the accuracy of the tools, little attention has been paid to the quality of the reports. Improving the report quality could improve the decision on possible false positives, shorten the bug fixing time and enhance the trust in static analysis tools. In this paper we report our research to find the most important attributes for generating clear and concise bug reports for the Clang Static Analyzer tool. With the help of experienced developers, we evaluated several test cases synthesized from real-world examples and analyzed how they rated the report elements according to the aspects of understanding. The results show that current reports from the Clang Static Analyzer, one of the most popular static analysis tools, can flood the developers with unimportant information while some of the report generation steps may eliminate relevant code parts. Our measurement methodology and results could be used to improve bug report quality, and therefore enhance the application of the tools. Despite our study focusing on one specific tool, the lessons learned could be used for a research targeting a wider range of static analyzers. Based on our findings, we made suggestions to the developers of the Clang Static Analyzer and an improvement to the bug report generation was made and is already available in version 19.0.0.