Evaluating Automated Unit Testing in Sulu

Sulu is a programming language designed with automated unit testing specifically in mind. One aim of Sulu is to demonstrate how automated software testing can be more integrated into current software development processes. Sulu's runtime and tools support automated testing from end to end; automating the generation, execution, and evaluation of test suites using both code coverage and mutation analysis. To show the effectiveness of this integrated approach, we performed an experiment to evaluate a family of test suites generated using a test case generation algorithm which exhaustively enumerates every sequence of method calls within a certain bound. The results show high code coverage, including 90% statement coverage and high mutation coverage for the most comprehensive test suite generated.