Integrating Handcrafted and Deep Features for Optical Coherence Tomography Based Retinal Disease Classification

Deep neural networks (DNNs) have been widely applied to the automatic analysis of medical images for disease diagnosis and to help human experts by efficiently processing immense amounts of images. While the handcrafted feature has been used for eye disease detection or classification since the 1990s, DNN was recently adopted in this area and showed a very promising performance. Since handcrafted and deep feature can extract complementary information, we propose, in this paper, three different integration frameworks to combine handcrafted and deep feature for optical coherence tomography image-based eye disease classification. In addition, to integrate the handcrafted feature at the input and fully connected layers using existing networks, such as VGG, DenseNet, and Xception, a novel ribcage network (RC Net) is also proposed for feature integration at middle layers. For RC Net, two “rib” channels are designed to independently process deep and handcrafted features, and another so-called “spine” channel is designed for the integration. While dense blocks are the main components of the three channels, sum operation is proposed for the feature map integration. Our experimental results showed that the deep networks achieved better classification accuracy after the integration of the handcrafted features, e.g., scale-invariant feature transform and Gabor. The RC Net showed the best performance among all the proposed feature integration methods.