Augmenting Diffs With Runtime Information

Source code diffs are used on a daily basis as part of code review, inspection, and auditing. To facilitate understanding, they are typically accompanied by explanations that describe the essence of what is changed in the program. As manually crafting high-quality explanations is a cumbersome task, researchers have proposed automatic techniques to generate code diff explanations. Existing explanation generation methods solely focus on static analysis, i.e., they do not take advantage of runtime information to explain code changes. In this paper, we propose Collector-Sahab, a novel tool that augments code diffs with runtime difference information. Collector-Sahab compares the program states of the original (old) and patched (new) versions of a program to find unique variable values. Then, Collector-Sahab adds this novel runtime information to the source code diff as shown, for instance, in code reviewing systems. As an evaluation, we run Collector-Sahab on 587 code diffs for Defects4J bugs and find it successfully augments the code diff for 90.6\% (532/587) of them. We also perform a manual analysis on 29 code diffs from 10 real-world projects and conclude that Collector-Sahab generates correct, causally relevant, and understandable augmented diffs. Overall, our experiments show the effectiveness and usefulness of Collector-Sahab in augmenting code diffs with runtime difference information. Collector-Sahab: https://github.com/algomaster99/collector-sahab