Deep Learning Based Integrated Classification and Image Retrieval System for Early Skin Cancer Detection

Skin cancer is one of the most frequent cancers among human beings. Diagnosing an unknown skin lesion is the first step to determine appropriate treatment. This paper proposes an integrated classification and retrieval based Decision Support System (DSS) for skin cancer detection with an `easy to use' user interface by applying fusion and ensemble techniques in deep feature spaces. The descriptiveness and discriminative power of features extracted from dermoscopic images are critical to achieve good classification and retrieval performances. In this work, several deep features are extracted based on using transfer learning in several pre-trained Convolutional Neural Networks (CNNs) and Logistic Regression and Support Vector Machine (SVM) models are built as ensembles of classifiers on top of these feature vectors. Furthermore, the content-based image retrieval (CBIR) technique uses the same deep features by fusing those in different feature combinations using a canonical correlation analysis. Based on image-based visual queries submitted by dermatologists, this system would respond by displaying relevant images of pigmented skin lesions of past cases as well as classifying the image category as different types of skin cancer. The system has been trained on a dermoscopic image dataset consists of 1300 images of ten different classes. The best classification (85%) and retrieval accuracies are achieved in a test data set when feature fusion and ensemble techniques are used in all available deep feature spaces. This integrated system would reduce the visual observation error of human operators and enhance clinical decision support for early screening of kin cancers.