DeepDTS: Diverse Test Selection Method for Deep Neural Networks

Deep Neural Networks (DNNs) are extensively ap-plied in data-driven modeling tasks. Testing these networks is critical due to their involvement in safety-critical applications. However, their black-box nature poses challenges in explaining their decisions and identifying incorrect behaviors. Detecting faults in a DNN requires selecting a diverse and error-revealing test set from a vast, unlabeled dataset. Diversity in the test set expands the coverage of the decision space of DNN, aiding in the identification of distinct faults. In this paper, we introduce Singular Entropy Density (SED) metric to evaluate the diversity of a test set on a DNN model. We demonstrate its effectiveness against other diversity measures. Additionally, we present DeepDTS, a diverse test selection method designed to identify misclassified inputs with higher diversity scores according to SED. To evaluate the effectiveness of DeepDTS relative to other coverage and uncertainty-based test selection methods, we conducted experiments with widely employed datasets and DNN models. Our results demonstrate that Deep-DTS outperforms other selection methods in terms of selecting a diverse test set.