Analysis on Various Approaches to Visualize and Interpret Convolution Neural Network

The paper aims to explore and evaluate the efficacy of methods that visualizes the internal structure of convolution neural networks as well as neural networks that are inherently interpretable. The paper implements three data set, including MNIST digits, MNIST Fashion, and CUB 200–2011. Tow base structures implemented to construct the convolution neural network are VGG 19 and a two-convolution layer CNN to provide intuitions upon analyzation. The paper toke the two approaches in analyzing the various methods: post hoc interpretation, which includes visualization of convolution kernels, and interpretable models, which includes a novel convolution neural network-based model. As for the outcomes, while the post hoc interpretations only offer superficial and unfaithful explanations to the CNN behavior, the use of the ProtoPNet in real-time interpretation provides an in-depth analysis of the model's reasoning process in decision making in contrast to the inexplicability of mainstream deep learning models.