Test Suite Generation Based on Context-Adapted Structural Coverage for Testing DNN

Deep learning (DL) introduces a novel data-driven programming paradigm, where the system logic is constructed through data training. However, this approach poses challenges in terms of system analysis and defect detection. To address this issue, recent efforts have focused on testing deep learning systems using intuitive criteria known as Neuron Coverage (NC) and its variants. These criteria measure the activation proportion of neurons in a neural network. Unfortunately, recent DL applications have largely overlooked the importance of context during testing. To address this gap, this paper first incorporates the context of the DL pipeline deployed before test execution, such as medical diagnosis and Android Malware detection. Next, we formulate structural coverage criteria to guide test suite generation based on the properties of DL pipeline in different contexts. Furthermore, we proposed a coverage-guided search algorithm to efficiently generate test suites. Experimental results highlight the effectiveness of our approach in uncovering numerous erroneous behaviors in contexts such as medical image diagnosis and Android malware detection. This approach significantly enhances the robustness of DL models by considering the structural aspects of the DL pipeline.