MaLT: Machine-Learning-Guided Test Case Design and Fault Localization of Complex Software Systems

Software testing is essential for the reliable and robust development of complex software systems. This is particularly critical for cyber-physical systems (CPS), which require rigorous testing prior to deployment. The complexity of these systems limits the use of formal verification methods. Furthermore, testing and fault localization can be very costly. To mitigate this cost, we outline in this work a holistic machine-learning-guided test case design and fault localization (MaLT) framework, which leverages recent probabilistic machine learning methods to accelerate the testing of complex software systems. MaLT consists of three steps: (i) the construction of a suite of test cases using a covering array for initial testing, (ii) the investigation of posterior root cause probabilities via a Bayesian fault localization procedure, then (iii) the use of such Bayesian analysis to guide selection of subsequent test cases via active learning. The proposed MaLT framework can thus facilitate efficient identification and subsequent diagnosis of software faults with limited test runs. This framework has potential for integration with an assertion-based test oracle approach, which may prove to be an efficient and cost-effective way of integrating light-weight formal methods with testing.