Multi-Modality Reconstruction Attention and Difference Enhancement Network for Brain MRI Image Segmentation

An important prerequisite for brain disease diagnosis is to segment brain tissues of Magnetic Resonance Imaging (MRI) into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). To improve performance, we propose a Multi-modality Reconstruction Attention and Difference Enhancement Network (MRADE-Net). We first stack three inputs of multiple MRI modalities along axial, sagittal, and coronal axes to form three enlarged volumes. Then we adopt global average pooling along each axis, fully connected layers, and activation functions to produce three orthogonal 2D coefficient maps, which are used to reconstruct a 3D attention map for weighting the three inputs. These weighted inputs are added together to generate a feature map of Multi-modality Reconstruction Attention (MRA). Similarly, we present Single-modality Reconstruction Attention (SRA) for improving feature representation abilities in middle stages. In addition, the difference of encoding features between adjacent layers is used to compensate the loss of spatial information caused by down-sampling. Experimental results show that the proposed approach is more effective than the existing state-of-the-art segmentation methods and its performance is verified on several datasets.