Dual-Branch Fully Convolutional Segment Anything Model for Lesion Segmentation in Endoscopic Images

The lesion area segmentation technology in endoscopic images helps doctors discover and locate lesion areas, which is crucial for improving the survival rate of patients. In various tasks within the field of medical image segmentation, the fully convolutional network with U-shaped architecture has achieved great success, but the fully convolutional operation has limitations in obtaining global context information.The Segment Anything Model (SAM), the latest foundational model in computer vision, boasts powerful image feature extraction capabilities and can capture explicit long-range dependencies, but it cannot capture low-level detailed information. This paper proposes a dual-branch model, named FCSAM, which makes up for the respective shortcomings of the fully convolutional network and SAM in medical image segmentation. At the same time, this paper proposes a LayerNorm and low-rank-based(LNLoRA) fine-tuning strategy, which overcomes the problem that large models cannot be applied to small-scale medical image datasets and reduces the deployment and storage overhead of the model. In addition, we also design a boundary aware module and a boundary key point map generation algorithm to improve the boundary localization ability of this model, and significantly improve the segmentation performance of lesions. The experimental results show that the IOU, Dice coefficient and accuracy of the model are 79.3%, 88.1% and 93.2% respectively. These metrics show that the model proposed in this paper achieves superior performance to various competing methods in terms of segmentation accuracy and has great potential in clinical analysis and diagnosis.