Fine-Grained Classification of Cervical Cells Using Morphological and Appearance Based Convolutional Neural Networks

Fine-grained classification of cervical cells into different abnormality levels is of great clinical importance but remains very challenging. Contrary to the traditional classification methods that rely on hand-crafted or engineered features, convolution neural network (CNN) can classify cervical cells based on automatically learned deep features. However, CNN in previous studies does not involve cell morphological information, and it is unknown whether morphological features can be directly modeled by CNN to classify cervical cells. This paper presents a CNN-based method that combines cell image appearance with cell morphology for classification of cervical cells in Pap smear. The training of cervical cell dataset consists of adaptively re-sampled image patches coarsely centered on the nuclei. Several CNN models (AlexNet, GoogLeNet, ResNet, and DenseNet) pre-trained on ImageNet dataset were fine-tuned on the cervical dataset for comparison. The proposed method is evaluated on the Herlev cervical dataset by five-fold cross-validation at patient-level splitting. The results show that by adding cytoplasm and nucleus masks as raw morphological information into appearance-based CNN learning, higher classification accuracies can be achieved in general. Among the four CNN models, GoogLeNet fed with both morphological and appearance information obtains the highest classification accuracies of 94.5%, 71.3%, and 64.5%, for two-class (abnormal versus normal), four-class (“The Bethesda System”), and seven-class (“World Health Organization classification system”) classification tasks, respectively. Our method demonstrates that combining cervical cell morphology with appearance information can provide improved classification performance. Although the initial results are promising, deep learning-based fine-grained cervical cell classification remains a very challenging task for a high-precision diagnosis.