Quadruple-Consistency Vision Transformer for Medical Image Segmentation with Limited Number of Sparse Annotations

Deep learning has significantly advanced the field of medical image segmentation but typically relies on extensive, densely annotated datasets, which are both costly and time-consuming to prepare. In response to the need for reducing annotation efforts, this study investigates a novel supervision approach named Semi-Scribble Supervised Learning, which utilizes a combination of semi-supervised (SSL) and weaklysupervised learning (WSL) techniques. This approach leverages both a large volume of unlabeled data and a smaller set of sparsely annotated, scribble-based labels. We introduce the Quadruple-Consistency Vision Transformer (4C-ViT), which capitalizes on the recent success of Vision Transformers in capturing intricate image features. Specifically, the proposed 4C-ViT employs an advanced consistency training strategy that incorporates quadruple perturbations at both the data and network levels, enhancing the network’s robustness and performance. The efficacy of 4 C -ViT is demonstrated on a publicly available MRI cardiac segmentation benchmark, where it outperforms other baseline methods across several evaluation metrics. The proposed $4 \mathrm{C}-\mathrm{ViT}$, alongside all baseline methods and the challenging yet realistic dataset, is made public available at https://github.com/ziyangwang007/CVSSL-MIS.