A Tailored Domain Analysis Method for the Development of System-Specific Testing DSLs Enabling Their Smooth Introduction in Automotive Practice

Automotive Test Case Specifications (TestSpecs) are a fundamental part of a structured test process in the automotive domain. For system and integration tests, acceptance and customer experience test cases are executed manually by human testers in a prototype vehicle. To ensure that these test cases are understood by humans, they are usually described in natural language, which often leads to ambiguities, misunderstandings, or incomplete test cases. In addition, the description of test cases vary significantly depending on the system to be tested and the respective test level. Test Designers want individual assistance in documenting their test cases with respect to system-specific characteristics, instead of using programming languages or standardized languages such as UML. Thus, Domain Specific Languages (DSLs) are a possible solution to satisfy this demand and to improve the quality of test cases, for example in terms of preciseness, uniformity, and completeness. The contribution of this paper is a systematic approach to support the development of system-specific automotive Testing DSLs that achieve high acceptance by test designers and testers. Therefore, we focus on the analysis phase in the DSL development process. We adapted domain analysis activities and defined a domain analysis method tailored to the analysis of automotive TestSpecs. We demonstrate the applicability of our method by means of five different automotive systems. Our evaluation shows that the derived system-specific Testing DSLs cover between 70% and 95% of the test steps contained in TestSpecs with only 11 to 35 conceptual templates. Moreover, a usability study with practitioners revealed a good usability of the Testing DSLs and the corresponding tool as well as that this eases the specification of test cases.