Semantic 3D Reconstruction Using Multi-View High-Resolution Satellite Images Based on U-Net and Image-Guided Depth Fusion

This paper reports our workflow for multi-view semantic stereo reconstruction in the IEEE Data Fusion Contest (DFC) of 2019. The workflow mainly consists of two parts: semantic classification and multi-view stereo digital surface model (DSM) generation. Our strategy involves the interaction of these two components for better classification and DSM results. We firstly classify semantic objects using only original image information and then project them to the generated DSM for voting and post-refinement. An example-based pair selection strategy is used to pre-selected pairs for dense matching, and the fused DSM is performed through an image-guided median filter. Since the adjusted rational polynomial coefficient (RPC) parameters are not accurate in the dataset, these DSMs are co-registered after production. In our workflow, we only consider non-vegetation and non-water area for registration to obtain more robust co-registration result. These co-registered DSMs are fused and refined using the result from the semantic classification. Our method achieved 73.00% of mIoU-3 in the testing data in the contest.