Reliability Evaluation of CNN-enabled Systems in Adversarial Scenarios

Convolutional Neural Network (CNN) usage has increased due to its ability to identify complex patterns and achieve high performance in many safety-critical domains, which face dynamically changing and actively hostile conditions characteristic of real-world applications, requiring these systems to be reliable. Many studies propose techniques to improve the robustness of these CNN algorithms. However, fewer consider quantitative techniques to assess changes in the reliability of these systems over time. This study demonstrates how to collect relevant data during the training and testing of CNN models suitable for applying software reliability models to track and predict defect discovery trends of CNN models in adversarial scenarios, providing ML and system engineers with an objective approach to compare the relative effectiveness of alternative training and testing methods. The approach is illustrated with an image classification model subjected to two generative adversarial attacks and then iteratively retrained to improve the system’s performance. Our results indicate that software reliability models incorporating covariates accurately characterized the misclassifi-cation discovery process, offering rigorous quantitative assurance methods for CNN-enabled systems.