Integrating Product Sampling and Behavioral Testing for Software Product Lines with Combinatorial Testing

Software Product Lines (SPLs) present a challenge in testing due to their configurable nature and variability. While traditional testing approaches focus on product enumeration based on feature models, they primarily assess product feasibility without adequate behavioral validation.In this paper, we introduce a novel testing strategy, CT4SPL, which integrates product sampling and behavioral testing, and it applies combinatorial interaction testing to both of them. Leveraging Feature Models and an extension of Featured Transition Systems, called Featured Transition Systems with Outputs, CT4SPL first samples the products with a desired interaction strength among features and then it generates test sequences covering all the t-way interactions between input events.We showcase the applicability of CT4SPL on a simple vending machine case study, and we show its efficacy by comparing it with a baseline approach which applies standard combinatorial techniques to product sampling and behavioral testing separately. With our experiments, we show how CT4SPL yields smaller yet more effective test suites with higher state and transition coverage.