Automatically recommending code reviewers based on their expertise: An empirical comparison

Code reviews are an essential part of quality assurance in Free, Libre, and Open Source Software (FLOSS) projects. However, finding a suitable reviewer can be difficult, and delayed or forgotten reviews are the consequence. Automating reviewer selection with suitable algorithms can mitigate this problem. We compare empirically six algorithms based on modification expertise and two algorithms based on review expertise on four major FLOSS projects. Our results indicate that the algorithms based on review expertise yield better recommendations than those based on modification expertise. The algorithm Weighted Review Count (WRC) recommends at least one out of five reviewers correctly in 69 % to 75 % of all cases, which is one of the best results achieved in the comparison.