Optimizing Model-based Generated Tests: Leveraging Machine Learning for Test Reduction

Several studies have shown Model-based Testing (MBT) as an efficient technique for generating fault-effective test cases. However, the automatic generation of test cases is compromised with redundant test cases providing no additional value to the coverage or fault detection effectiveness while impacting test execution efficiency, especially, in a dynamic development environment where providing timely feedback is crucial. These redundant test cases need to be discarded to minimize the test suite size and their effect on the execution cost and efficiency of a test suite. Reducing a test suite becomes challenging for black box testing at the system level when no information regarding the coverage and fault detection effectiveness of the test suite exists. Hence, in this paper, we have presented a test suite optimization approach leveraging different machine learning algorithms, a greedy algorithm, and a similarity measure. The proposed approach generates a reduced test suite by identifying and eliminating redundant test cases from an MBT-generated test suite while having minimal impact on the fault detection rate. We have also performed a comparative evaluation of the optimized test suites with the MBT-generated and manually created test suites in terms of fault detection effectiveness and test execution efficiency using an industrial case study from Alstom Rail AB, Sweden. The results show a significant reduction of $85 \%$ to $\mathbf{9 2 \%}$ in the size of the test suite. Moreover, we also found the test execution time of the optimized test suite equivalent to the manually created tests and a fault detection rate within the range of $95 \%$ to $100 \%$ for all test suites under observation.