Executable Test Case Generation from Specifications Written in Natural Language and Test Execution Environment

The Software Product Line Engineering (SPLE) realizes various products, reusing software parts, whereas issues remain in test case design and execution. Test cases are conventionally designed by a manual routine from specifications written in a natural language, and the routine and redesign of the test cases caused by the defects in the specification require much human time. Also, functions of recent consumer products are invoked in non-deterministic order by messages sent over a network, and combinations of software parts and execution orders require many regression tests, which are time-consuming and often infeasible to execute manually due to limited development time. Against the above issues, we introduce a test design process for specifications written in natural language, support tools for the process, and a test execution environment that automatically executes the non-deterministic tests to reduce the human time of both test case design and execution. Case studies confirmed that the proposing process automated the manual routine, removed defects in a specification, and generated test cases. The case studies also showed that the test execution environment automatically executed the non-deterministic tests for an HVAC system developed with the SPLE. Finally, we confirmed that the proposing methods shortened the human time of design and execution of tests.