Visualizing Association in Exemplar-Based Classification

Recent progress in deep learning has enhanced image classification performance. However, classification using deep convolutional neural networks lacks interpretability. To solve this problem, we propose a novel method of explainable classification; this method uses images representing each image class, which we call exemplars. Our method comprises encoder-decoder models (association networks) and a classifier. First, the association networks transform each input image into an image that a deep neural network associates, which we call an associative image. Then, the image-level similarity between the associative images and the exemplars is used as a feature for classification. This similarity explains the decision of the classifiers. We conducted experiments using CIFAR-10, CIFAR-100, and STL-10 and demonstrated our classifier’s interpretability through the proposed visualization technique.