Learning Discriminative Global and Local Features for Building Extraction from Aerial Images

Buildings constitute one of the most important landscapes in remote sensing images. Automatic building extraction methods towards the very high resolution remote sensing imagery feature both the local refinement of segmentation results and the context-aware reasoning for segmentation. In this paper, we propose a novel dual-stream convolutional neural network (DS-Net) to collaboratively incorporate local and global features for accurately segmenting buildings in very high resolution aerial images. We develop a hierachical representation and a deep feature sharing strategy for both the local branch and global branch in DS-Net to effectively exploit the complementarity between the two branches. Through extensive experiments on the large-scale building detection datasets, we show that the proposed DS-Net can benefit from both the local and global features, which significantly improves the accuracy of building extraction over diversified remote sensing scenes.