A Novel Approach on Segmentation of AgNOR-Stained Cytology Images Using Deep Learning

Cervical cancer is the second most common cancer type in women. This is a deadly disease that could benefit from early detection methods. Cytology is a possible, noninvasive, alternative to biopsy for diagnosis of malignant lesions, where information about the cells can be obtained. This process consists of staining the slides being able to visualize the nuclei of the cells and then be properly diagnosed. In this work, silver-stained slides, a method known as Argyrophilic Nucleolar Organizer Regions (AgNOR), is employed. This method shows great potential for the diagnosis of the malignancy of lesions. However, it has not yet been much explored, mainly using computational methods. In this work, we propose the use of Convolutional Neural Networks on silver stained images to measure AgNOR quantitative image information. We will present the entire process, starting from pre-processing, passing through neural network training and results evaluation. The best results we obtained were 0.87 of Mean Intersection Over Union (mIoU) and 0.99 of Dice Similarity Coefficient (DSC) using an U-Net model with a ResNet18 as its backbone. The code and the dataset employed in this work are publicly available.