Disposable Testing: Avoiding Maintenance of Generated Unit Tests by Throwing Them Away

Developers write unit tests together with program code, and then maintain these tests as the program evolves. Since writing good tests can be difficult and tedious, unit tests can so be generated automatically. However, maintaining these tests(e.g., when APIs change, or, when tests represent outdated and changed behavior), is still a manual task. Because automatically generated tests may have no clear purpose other than covering code, maintaining them may be more difficult than maintaining manually written tests. Could this maintenance be avoided by simply generating new tests after each change, and disposing the old ones? We propose disposable testing: Tests are generated to reveal any behavioral differences caused by a code change, and are thrown away once the developer confirms whether these changes were intended or not. However, this idea raises several research challenges: First, are standard automated test generation techniques good enough to produce tests that may berelied upon to reveal changes as effectively as an incrementally built regression test suite? Second, does disposable testing reduce the overall effort, or would developers need to inspect more generated tests compared to just maintaining existing ones?