A Multi-scale Feature Hierarchical Aggregation Network for Neurosurgical scene Segmentation

Neurosurgical scene segmentation can provide surgeons with the real-time location and perimeter details of important tissues and instruments. The anatomical structure representation of surgical instruments and tissues can contribute to evaluating surgical skills, tracking surgical instruments and carrying out intraoperative navigation. However, some inherent limitations including the scale variability, local similarity and interclass similarity of instruments and tissues, bring bottlenecks for accurate segmentation. To address these challenges, we propose a Multi-Scale Feature Hierarchical Aggregation Network (MFHA-Net). Specially, attention-based multi-scale feature hierarchical fusion module (AMFHF), multi-layer feature aggregation and high-level semantic guidance module (MFAASG) are designed to deal with the scale change and similarity of surgical instruments and tissues. We use pre-trained Resnet50 as the feature extraction backbone to obtain channel and spatial semantic features, while The AMFHF module is employed to enhance the features of different scales using the attention mechanism, enabling adaptation to scale changes across various targets. In addition, AMFHF module combines multi-scale features with different receptive fields, using high-layer semantic information to guide low-layer feature transmission, which can improve the model's feature discriminative ability. A neurosurgical dataset named Neuro including a variety of neurosurgical surgical instruments and brain anatomy is constructed. The proposed network in this paper achieves the best performance on the Neuro dataset, attaining the highest accuracy metrics of 68.72% MIoU, 80.09% Dice, 81.92% aAcc, and 88.32% mAcc.