Automatic Program Repair as Semantic Suggestions: An Empirical Study

Automated Program Repair (APR) is an area of research focused on the automatic generation of bug-fixing patches. Current APR approaches present some limitations, namely overfitted patches and low maintainability of the generated code. Several works are tackling this problem by attempting to come up with algorithms producing higher quality fixes. In this experience paper, we explore an alternative. We believe that by using existing low-cost APR techniques, fast enough to provide real-time feedback, and encouraging the developer to work together with the APR inside the IDE, will allow them to immediately discard proposed fixes deemed inappropriate or prone to reduce maintainability. Most developers are familiar with real-time syntactic code suggestions, usually provided as code completion mechanisms. What we propose are semantic code suggestions, such as code fixes, which are seldom automatic and rarely real-time. To test our hypothesis, we implemented a Visual Studio Code extension (named pAPRika), which leverages unit tests as specifications and generates code variations to repair bugs in JavaScript. We conducted a preliminary empirical study with 16 participants in a crossover design. Our results provide evidence that, although incorporating APR in the IDE improves the speed of repairing faulty programs, some developers are too eager to accept patches, disregarding maintenance concerns.