Non-transfer Deep Learning of Optical Coherence Tomography for Post-hoc Explanation of Macular Disease Classification

Deep transfer learning is widely used for medical image classification by leveraging models that are pretrained by natural images. This choice may introduce unnecessary model complexity that can limit explanations of such model outcomes in clinical practice. To investigate this hypothesis, we develop a configurable deep convolutional neural network (CNN) to classify four macular disease types using retinal optical coherence tomography (OCT) images. Our proposed non-transfer deep CNN model (acc: 97.9%) outperforms existing transfer learning models such as ResNet-50 (acc: 89.0%), ResNet-101 (acc: 96.7%), VGG-19 (acc: 93.3%), and Inception-V3 (acc: 95.8%) in the same retinal OCT image classification task. Our post-hoc analysis of the model extracted image features reveals that only eight out of 256 CNN filter kernels are active at the final convolutional layer. The convolutional responses of these eight selective filters yield image features that efficiently separate four macular disease classes even when projected onto two-dimensional principal component space. Our findings suggest that a large portion of deep learning parameters and computations are redundant for retinal OCT image classification, which intensifies when using transfer learning. Additionally, we provide clinical interpretations of our misclassified test images identifying manifest artifacts, shadowing of useful texture, false texture representing fluids, and other confounding factors. These clinical explanations along with model optimization via filter selection can improve the classification accuracy, computational costs, and explainability of deep model outcomes.