Semantic Alignment Network For Building Extraction From Remote Sensing Images

Recently, the Vision Transformer (ViT) becomes a promising model for building extraction from remote sensing images. However, most of these methods ignore the misalignment issue caused by the semantic inconsistency between different layers. To settle this issue, we propose a semantic alignment network (SANet) for building extraction. First, a dense prediction network with the encoder-decoder architecture is developed. The encoder is constituted by stacking multiple ViT blocks to extract multi-level features. Then, the semantic alignment module (SAM) is designed as the decoder to align and fuse low-level and high-level features, by use of semantic flow field estimation. The semantic flow assesses discrepancy between features of adjacent layers to find a flow field that guide the alignment between two-layer feature maps effectively. By this manner, the semantic gap can be narrowed. Experimental results on the two well-known datasets demonstrate the effectiveness of the proposed method.