The Impact of List Reduction for Language Agnostic Test Case Reducers

To find and fix bugs in compilers or in other code processing tools, modern language agnostic test case reducers boil the input files down to small bug-triggering versions. To do so they carefully craft lists of potentially irrelevant items and apply a list reduction to minimize them. We show that substituting the chosen list reduction algorithm improves the overall reducer runtime without affecting the final file sizes much. In a comparative study we combine 6 renowned test case reducers with 7 established list reductions. Most renowned reducers become faster by switching to another list reduction. We also present three ways to preprocess the crafted lists before the test case reducers pass them to the list reductions. We discuss the conditions for the preprocessings to improve the reducers' speeds even more. On a benchmark of 321 C and SMT-LIB2 compiler bugs, selecting a different list reduction saves up to 74.7% of the runtime. Most test case reducers benefit from such a substitution. Preprocessing saves up to 9.1 additional percentage points. Combining these ideas saves up to 75.2% of the runtime.