CT-PromptSAM: Collaborative Prompting With Hybrid CNN–Transformer for Remote Sensing Semantic Segmentation

Remote sensing (RS) semantic segmentation (SS) faces critical challenges in small-object detection and boundary precision due to scale variation and complex scenes. This letter proposes CT-PromptSAM, a specialist–generalist framework integrating a CNN–Transformer specialist network and an SAM-adaptive generalist network. The specialist network employs a tandem encoder for local–global feature fusion, coupled with three decoder modules: upsampling feature refinement module (UFRM), multiscale feature interaction module (MFIM), and mutual-guided semantic-spatial attention module (MSSAM), to generate task-adaptive masks for small-class enhancement. The generalist network adapts segment anything model (SAM) to RS via multimodal prompts, forming a “local–global boundary” collaborative loop for boundary refinement. On the Vaihingen and Potsdam datasets, CT-PromptSAM achieves state-of-the-art (SOTA) small-class IoU and mean Boundary $F1$ -score, outperforming competitors with comparable parameter efficiency. By fusing RS-specific semantic priors (from the specialist) and SAM’s geometric precision (from the generalist), the framework balances small-object and boundary tasks, validating its efficacy in complex RS scenes. Our code is publicly available at https://github.com/WangXin81/CT-PromptSAM/