BuildingSAM: A Dual-Branch Feature-Augmented Segment Anything Model for Remote Sensing Building Extraction

We propose a segment anything model for building (BuildingSAM) extraction as a general solution for building extraction (BE) from high-resolution remote-sensing images. Unlike previous methods, BuildingSAM is constructed based on the segment anything model (SAM) for large-scale images and parameter-efficient fine-tuning (PEFT), representing a novel research paradigm for BE. Although transformer-based architecture excels at processing global and low-frequency information, it can overlook local details and introduce biases in feature learning. To address these shortcomings, BuildingSAM incorporates a dual-branch module, in which one branch employs the image encoder of the lightweight next-generation semantic segmentation network SegNeXt to focus on capturing local building details, and the other uses a vision transformer (ViT) image encoder to extract global features. Furthermore, we introduce the Conv-low-rank adaptation (LoRA) method, which integrates ultra lightweight convolutional parameters into LoRA to inject image-related inductive biases into the ViT image encoder. This approach enhances the ability of BuildingSAM to learn building boundary features in complex regions efficiently. In experiments conducted using two BE benchmark datasets, the proposed method significantly outperformed the existing state-of-the-art (SOTA) BE models. Comprehensive ablation studies further validated the superior performance of BuildingSAM, at minimal additional computational cost.