Augmenting Diffs With Runtime Information

Source code diffs are used on a daily basis as part of code review, inspection, and auditing. To facilitate understanding, they are typically accompanied by explanations that describe the essence of what is changed in the program. As manually crafting high-quality explanations is a cumbersome task, researchers have proposed automatic techniques to generate code diff explanations. Existing explanation generation methods solely focus on static analysis, i.e., they do not take advantage of runtime information to explain code changes. In this article, we propose
Collector-Sahab
, a novel tool that augments code diffs with runtime difference information.
Collector-Sahab
compares the program states of the original (old) and patched (new) versions of a program to find unique variable values. Then,
Collector-Sahab
adds this novel runtime information to the source code diff as shown, for instance, in code reviewing systems. As an evaluation, we run
Collector-Sahab
on
584
code diffs for Defects4J bugs and find it successfully augments the code diff for 95% (555/
584
) of them. We also perform a user study and ask eight participants to score the augmented code diffs generated by
Collector-Sahab
. Per this user study, we conclude that developers find the idea of adding runtime data to code diffs promising and useful. Overall, our experiments show the effectiveness and usefulness of
Collector-Sahab
in augmenting code diffs with runtime difference information.
Publicly-available repository:

https://github.com/ASSERT-KTH/collector-sahab
.