Safely Evolving Preprocessor-Based Configurable Systems

Since the 70s, the C preprocessor is still widely used in practice in a numbers of projects, including Apache, Linux, and Libssh, to tailor systems to different platforms. To better understand the C preprocessor challenges, we conducted 40 interviews and a survey among 202 developers. We found that developers deal with three common problems: configuration-related bugs, combinatorial testing, and code comprehension. To safely evolve preprocessor-based configurable systems, we proposed strategies to detect preprocessor-related bugs and bad smells, and a set of 16 refactorings to remove bad smells. To better deal with exponential configuration spaces, we compared 10 sampling algorithms with respect to effort (i.e., number of configurations to test) and bug-detection capabilities (i.e., number of bugs detected in the sampled configurations). Based on the results, we proposed a sampling algorithm with a useful balance between effort and bug detection capability. By evaluating the proposed solution using 40 popular projects, we found 131 preprocessor-related bugs and more than 5K opportunities to apply the refactorings in practice.