Research on Data Sufficiency and Model Validity toward AI for Critical Systems

Deep learning is used in many fields, but it has not been used in critical systems because testing that it works correctly is difficult. There are no comprehensive methods to ensure deep learning quality. Our research introduced effective test methods for deep learning in terms of the data sufficiency and model validity based on user requirements. To determine the data sufficiency, we proposed a method using the new concept of extrapolation and interpolation to classify out-of-domain uncertainty, domain-shift uncertainty, in-domain uncertainty, and data uncertainty. This method allows us to indicate what training data are missing. This paper proposes methods to visualize a basis for the model's decisions to determine the model's validity. We explain how to visualize the region of interest in an image and explain the basis for deep learning decisions from only human-understandable explanations using the combination of intermediate output and a decision tree.