Transparent Embedding Space for Interpretable Image Recognition

When humans explain their reasoning, such as their classification decisions, they often break down an image into parts and highlight the evidence from those parts to support the concepts they have in mind. Drawing inspiration from this cognitive process, several self-explaining models have been proposed to explain predictions by part-level concepts. However, these models can be limited by their structure and difficulty in determining the effect of individual parts on the output category. To address these challenges, we introduce a self-explaining architecture that uses a plug-in transparent embedding space (TesNet) to connect high-level input patches (e.g. feature maps or tokens) with output categories. The transparent embedding space is spanned by basis concepts and constructed on the Grassmann manifold. The basis concepts are enforced to be category-aware, and within-category concepts are orthogonal to each other, ensuring the embedding space is disentangled. To reduce concept redundancy and restore the concept space structure, we introduce two concept pruning methods and a new re-training strategy to build a slimming transparent embedding space. We verify the scalability of TesNet through experiments on deep networks such as VGG, ResNet, DenseNet, and Vision Transformer. Additionally, we design several metrics for self-explaining models to quantify interpretability and compare them with state-of-the-art self-explaining methods. Our experiments demonstrate that TesNet is much more effective for classification tasks, providing better interpretability on predictions and improving final accuracy.