MMFC-ART: a Fixed-size-Candidate-set Adaptive Random Testing approach based on the modified Metric-Memory tree

Adaptive random testing (ART) improves the failure-detection effectiveness of Random testing (RT) by making test cases more evenly distributed in the input domain. The Fixed-size-Candidate-set ART (FSCS-ART) is one of the most classical algorithms, which selects the candidate test case furthest from the previously executed test case as the next test case. However, when the number of executed test cases is large, the computational overhead will be very high. In this paper, we propose an enhanced version of FSCS-ART based on a modified Metric-Memory tree (MM-tree), namely Fixed-size-Candidate-set ART based on the modified MM-tree (MMFC-ART). Simulations and empirical studies are conducted to verify the effectiveness and efficiency of MMFC-ART. The experimental results indicate that MMFC-ART significantly reduces the computational overhead while ensuring comparable or better failure-detection effectiveness than FSCS-ART. Meanwhile, compared with KD-tree-enhanced Fixed-size-Candidate-set ART (KDFC-ART), MMFC-ART has better performance in high dimensions in terms of efficiency. In terms of effectiveness, MMFC-ART has better failure-detection effectiveness in some scenarios. Overall, MMFC-ART is cost-effective compared to FSCS-ART and KDFC-ART.

I. Introduction

Software testing is one of the important techniques to detect program failures and ensure the quality of the software. Random testing (RT) [1], as a black-box testing method, detects software failures by randomly generating test cases in the input domain. Due to its simplicity and efficiency, RT has been widely used in many different testing scenarios and systems, including embedded software systems [2], Mac OS robustness assessment [3], SQL database systems [4], .NET error detection [5] and Android applications [6].