Multi-Modal Magnetic Resonance Images Segmentation Based on An Improved 3DUNet

The 3D U-Net model employs end-to-end training ways and does not demand pre-training process, but the limited acceptance range of convolutional kernel makes it difficult to establish an explicit long-range dependency, resulting in poor segmentation accuracy in magnetic resonance (MR) image. This paper presents an promoted 3D U-Net architecture that incorporates the Transformer in 3D U-Net (Trans3DUNet) to segment multi-modal MR images, called MMTrans3DUNet. Firstly, the tokenized image blocks from a convolutional neural network (CNN) feature mapping are encoded by Transformer as the input sequence to extract the global context. Then, the decoder up-sampling the encoded features and coalesce them in CNN feature mapping with high resolution to achieve exact positioning. Moreover, according to the characteristics of MR images with multiple imaging modes, the four modalities images (t l, t lce, t2, flair) are fused and put into the Trans3DUNet model for training, which can overcome the problem that the single-modal MR image cannot sufficiently subdivide the lesion in the relevant area. The experimental results on the BraTS2018 and BraTS2019 dataset show that MMTrans3DUNet model can further promote the efficiency and precision of segmentation due to the image information of multiple modes which can complement each other.