Rethinking Semantic Change Detection From a Semantic Alignment Perspective

Recent advancements in semantic change detection (SCD) often divide the task into two subtasks: semantic segmentation (SS) and binary change detection (BCD), typically employing three decoding heads optimized jointly based on their respective losses. However, this approach faces challenges in aligning semantic feature spaces across temporal domains, complicating the construction of change features. In this article, we propose the novel PRO-HRSCD framework, which utilizes two decoding heads: one dedicated to BCD and another for the SS subtask. The SS subtasks for bi-temporal images are unified within a prototype-based SS (PRO-SS) decoding head, which aligns semantic features across temporal domains using online prototype learning rather than relying solely on joint optimization. The prototype-based contrastive loss targets the semantic features themselves, effectively addressing large intraclass variance and bridging the feature gap between temporal domains. Additionally, a temporal correlation exploration module (TCEM) is integrated into the BCD head, leveraging the highest resolution features of bi-temporal images encoded by HRNet to enhance temporal dependency modeling, leading to mutual reinforcement with PRO-SS. The experimental results on two benchmark datasets demonstrate that our method surpasses state-of-the-art (SOTA) SCD approaches. Our source code is available at https://github.com/sdust-mmlab/PRO-HRSCD