SAM-LSCI: Leveraging Local Spatial Context Information to Refine Image Segmentation

The Segment Anything Model (SAM) has manifested remarkable strides within the domain of image segmentation. Nonetheless, SAM grapples with two predominant challenges in specific scenarios: the emergence of "mask holes" and the coarse boundaries of the masks. To ameliorate these issues, we have proposed a architecture named SAM with Local Spatial Context Information (SAM-LSCI). This architecture ingeniously amalgamates and reutilizes the pre-trained SAM structure to elevate the segmentation quality while maintaining zero-shot image segmentation performance. We have appended a Local Spatial Feature Enhancement Module, predicated on Convolutional Neural Networks, to the original image encoder of SAM to compensate for the deficiencies of ViT in capturing image details. Furthermore, we introduced a learnable Global Context Token to augment the masks' quality further. The qualitative and quantitative experimental results on the HQSeg-44K dataset unveil that, compared to SAM and HQ-SAM, SAM-LSCI can generate masks of superior quality.