Dataset Fault Tree Analysis for Systematic Evaluation of Machine Learning Systems

Recently, machine learning, particularly deep learning, is attracting much interest and is applied in various systems. Applications include not only entertainment systems, but safety-critical systems such as those found in autonomous vehicles. The reliability of such safety-critical systems must be guaranteed before they are released into society. However, methods for ensuring the safety of machine learning-based systems have yet to be established. In this paper, we propose a method for systematically evaluating the safety of such systems. The method consists of dataset-based safety analysis and statistical evaluation of testing results. In the safety analysis, we extend the widely used fault tree analysis to deal with datasets. In the testing, we use statistical estimation to guarantee recognition rates obtained in the safety analysis. We conducted experiments using a handwritten character recognition system implemented as a CNN to demonstrate the feasibility and effectiveness of our method.