RMHA-Net: Robust Optic Disc and Optic Cup Segmentation Based on Residual Multiscale Feature Extraction With Hybrid Attention Networks

Glaucoma is a chronic eye disease that damages the optic nerve, often leading to permanent vision loss. Early screening with automated technology is crucial to assist ophthalmologists in making accurate diagnoses. One of the key technologies for automated diagnosis is the segmentation of the optic disc (OD) and optic cup (OC). In this paper, RMHA-Net is developed using a residual multiscale feature extractor with a hybrid attention mechanism, which is introduced purposely for automated OD and OC segmentation. This network’s encoder is designed based on advanced convolutional neural network (CNN) blocks that combine dilated convolution, which allows field-of-view expansion by capturing larger-scale features. In addition, the encoder also embeds residual connections to improve the model capacity in extracting low-level features. This design accurately separates the OD and OC from surrounding retinal tissues, handling complex environmental and anatomical changes. The proposed network is further improved by integrating two modules to enhance the segmentation performance: 1) a multiscale feature extractor module to provide various scales contextual information, and 2) dual attention mechanisms through channel-wise and spatial-wise mechanisms so that irrelevant information or noise can be excluded by assigning lesser weights to irrelevant features. To validate RMHA-Net’s effectiveness, extensive experiments were conducted using five public datasets: Drishti-GS, ORIGA, PAPILA, Chaksu, and REFUGE, and one private dataset, Ibn Al-Haitham. The proposed model outperformed seven cutting-edge segmentation models for OD and OC segmentation. The results demonstrate that the network extracts detailed features, offering an efficient framework for future studies.