Automated Testing of DSL Implementations - Experiences from Building mbeddr

Domain specific languages promise to improve productivity and quality of software by providing problem-adequate abstractions to developers. Projectional language workbenches like JetBrains MPS allow the definition of modular and extensible domain specific languages, generators and development environments. While recent advances in language engineering have enabled the definition of DSLs and tooling in a modular and cost-effective manner, the quality assurance of their implementation is still challenging. In this paper we present our work on testing the implementation of domain specific languages and associated tools, and discuss different approaches to increase the automation of language testing. We illustrate this based on MPS and our experience with testing mbeddr, a set of domain specific languages and tools on top of C tailored to embedded software development.