CT-PromptSAM: Collaborative Prompting with Hybrid CNN-Transformer for Remote Sensing Semantic Segmentation

Remote sensing (RS) semantic segmentation (SS) faces critical challenges in small-object detection and boundary precision due to scale variation and complex scenes. This letter proposes CT-PromptSAM, a specialist-generalist framework integrating a CNN-Transformer specialist network and a SAM-adaptive generalist network. The specialist network employs a tandem-encoder for local-global feature fusion, coupled with three decoder modules: Up-sampling Feature Refinement Module (UFRM), Multi-scale Feature Interaction Module (MFIM), and Mutual-guided Semantic-Spatial Attention Module (MSSAM), to generate task-adaptive masks for small-class enhancement. The generalist network adapts Segment Anything Model (SAM) to RS via multi-modal prompts, forming a “local-global-boundary” collaborative loop for boundary refinement. On Vaihingen and Potsdam datasets, CT-PromptSAM achieves state-of-the-art small-class IoU and mean Boundary F1-score, outperforming competitors with comparable parameter efficiency. By fusing RS-specific semantic priors (from the specialist) and SAM’s geometric precision (from the generalist), the framework balances small-object and boundary tasks, validating its efficacy in complex RS scenes. Our code is publicly available at: https://github.com/WangXin81/CT-PromptSAM/.