An Evaluation of Model-Based Testing in Embedded Applications

Testing is one of the most important quality assurance techniques for software. Automating the test design allows for automatically creating test suites from high-level system descriptions or test descriptions. Furthermore, it enables testers to automatically adapt the test suites to potentially recently changed descriptions. In model-based testing, models are used to automatically create test cases. Case studies report of an effort reduction in test design between 20 and 85 percent. In this paper, we report on a pilot project for introducing model-based testing in an industrial context. For such a pilot project, it is necessary to adapt the existing workflows and tool chains, to train the staff, and to adapt the assets of the company. The goal is to show the full applicability of the technique at the customer site. We present the evaluations, the lessons learned, and compare the efforts of model-based and manual test design for this example. This paper is not 'generally valid' in the sense that the results are reproducible for other projects and domains. Instead, our intention is to provide guidance for setting up similar evaluations.