Exploring the Role of Automation in Duplicate Bug Report Detection: An Industrial Case Study

Duplicate bug reports can increase technical debt and tester workload in long-running software projects. Many automated techniques have been proposed to detect potential duplicate reports. However, such techniques have not seen widespread industrial adoption. Our objective in this study is to better understand how automated techniques could effectively be employed within a tester’s duplicate detection workflow. We are particularly interested in exploring the potential of a human-in-the-loop scenario where tools and humans work together to make duplicate determinations. We have conducted an industrial case study where we characterize the current tester workflow. Based on this characterization, we have developed Bugle-an automated technique based on a complex language model that suggests potential duplicates to testers based on an input bug description that can be freely reformulated if the initial suggestions are irrelevant. We compare the assessments of Bugle and testers of varying experience, capturing how often-and why-opinions might differ between the two, and comparing the strengths and limitations of automated techniques to the current tester workflow. We additionally examine the influence of knowledge and biases on accuracy, the suitability of language models, and the limitations affecting duplicate detection techniques. CCS CONCEPTS • Software and its engineering → Software post-development issues; Collaboration in software development; Software verification and validation; Software maintenance tools.