MA-SAM: A Multi-atlas Guided SAM Using Pseudo Mask Prompts without Manual Annotation for Spine Image Segmentation

Accurate spine segmentation is crucial in clinical diagnosis and treatment of spine diseases. However, due to the complexity of spine anatomical structure, it has remained a challenging task to accurately segment spine images. Recently, the segment anything model (SAM) has achieved superior performance for image segmentation. However, generating high-quality points and boxes is still laborious for high-dimensional medical images. Meanwhile, an accurate mask is difficult to obtain. To address these issues, in this paper, we propose a multi-atlas guided SAM using multiple pseudo mask prompts for spine image segmentation, called MA-SAM. Specifically, we first design a multi-atlas prompt generation sub-network to obtain the anatomical structure prompts. More specifically, we use a network to obtain coarse mask of the input image. Then atlas label maps are registered to the coarse mask. Subsequently, a SAM-based segmentation sub-network is used to segment images. Specifically, we first utilize adapters to fine-tune the image encoder. Meanwhile, we use a prompt encoder to learn the anatomical structure prior knowledge from the multi-atlas prompts. Finally, a mask decoder is used to fuse the image and prompt features to obtain the segmentation results. Moreover, to boost the segmentation performance, different scale features from the prompt encoder are concatenated to the Upsample Block in the mask decoder. We validate our MA-SAM on the two spine segmentation tasks, including spine anatomical structure segmentation with CT images and lumbosacral plexus segmentation with MR images. Experiment results suggest that our method achieves better segmentation performance than SAM with points, boxes, and mask prompts.