MSSFC-Net: Enhancing Building Interpretation With Multiscale Spatial–Spectral Feature Collaboration

Building interpretation from remote sensing imagery primarily involves two fundamental tasks: building extraction (BX) and change detection (CD). However, most existing methods address these tasks independently, overlooking their inherent correlation and failing to exploit shared feature representations for mutual enhancement. Furthermore, the diverse spectral, spatial, and scale characteristics of buildings pose additional challenges in jointly modeling spatial–spectral multiscale features and effectively balancing precision and recall. The limited synergy between spatial and spectral representations often results in reduced detection accuracy and incomplete change localization. To address these challenges, we propose a multiscale spatial–spectral feature cooperative dual-task network (MSSFC-Net) for joint BX and CD in remote sensing images. The framework integrates both tasks within a unified architecture, leveraging their complementary nature to simultaneously extract building and change features. Specifically, a dual-branch multiscale feature extraction (DMFE) module with spatial–spectral feature collaboration (SSFC) is designed to enhance multiscale representation learning, effectively capturing shallow texture details and deep semantic information, thus improving BX performance. For temporal feature aggregation, we introduce a multiscale differential fusion module (MDFM) that explicitly models the interaction between differential and dual-temporal features. This module refines the network’s capability to detect large-area changes and subtle structural variations in buildings. Extensive experiments conducted on three benchmark datasets demonstrate that MSSFC-Net achieves superior performance in both BX and CD tasks, effectively improving detection accuracy while maintaining completeness.