An Efficient Spatial Modeling Conv-ViT using Mask Supervision for 3D Medical Image Segmentation

Transformer-CNN hybrid models and ConvNeXt-based models have demonstrated notable success in 3D medical image segmentation. However, these models have redundancy problems with a large number of parameters and fall short of capturing semantic information effectively, resulting in high modeling costs and limitations on segmentation performance. Motivated by this, we propose an efficient spatial modeling Convolutional Vision Transformer using Mask Supervision. We design a convolutional encoding block embedded with global context information for 3D medical images so that the inductive bias of CNN and the powerful global modeling capability of the Transformer are effectively integrated. We also adopt the structure of 3D Dynamic Convolution to alleviate the problem of anisotropy of features in medical images. In the Mask Supervision strategy, we acquire the semantic mask after each stage of the encoder through an efficient attention mechanism. The semantic mask is employed for supervision in model training, enhancing the model’s capacity to assimilate more intricate semantic information. We conducted experiments on two authoritative abdominal multi-organ segmentation datasets, AMOS2022 and WORD. The experimental results show that our model outperforms all competitive models on both datasets(AMOS2022:90.00, WORD:86.28) with fewer parameters(32.53M), demonstrating the high modeling efficiency and effectiveness of our methods.