Amplifying Integration Tests with CAMP

Modern software systems interact with multiple 3rd party dependencies such as the OS file system, libraries, databases or remote services. To verify these interactions, developers write so-called "integration tests" that exercise the software within a specific environment. These tests are not only difficult to write as their environment is complicated, but they are also brittle because changes outside the code (i.e., in the environment) might make them fail unexpectedly. Integration tests are thus underused whereas they could help find many more issues. We hence propose CAMP, a tool that amplifies an existing integration test by exploring variations of the given environment. The tests that CAMP generates alter the services orchestration, the software stacks, the individual components' configuration or any combination thereof. We used CAMP to amplify tests from the Sphinx and Atom open-source projects, and in both cases, we were able to spot undocumented issues related to incompatible environments.