Toward Prototypical Part Interpretable Similarity Learning With ProtoMetric

The Prototypical Part Network (ProtoPNet) is an interpretable deep learning model that combines the strong power of deep learning with the interpretability of case-based reasoning, thereby achieving high accuracy while keeping its reasoning process interpretable without any additional supervision. Thanks to these advantages, ProtoPNet has attracted significant attention and spawned many variants that improve both the accuracy and the computational efficiency. However, since ProtoPNet and its variants (ProtoPNets) adopt a training strategy specific to linear classifiers or decision trees, they run into difficulty when utilized for similarity learning, which is a practically useful technique for cases in which unknown classes exist. To solve this problem, we propose ProtoMetric, an extension of ProtoPNet that is applicable to similarity learning. Extensive experiments on multiple open datasets for fine-grained image classification demonstrate that ProtoMetric achieves a similar accuracy as state-of-the-art ProtoPNets with a smaller number of prototypes. We also demonstrate through case studies that ProtoMetric is applicable to image retrieval tasks where the class labels of the training and test sets are completely different.