Weakly Supervised 3-D Building Reconstruction From Monocular Remote Sensing Images

3-D building reconstruction from monocular remote sensing imagery is an important research problem that has been extensively studied for several decades. Although monocular remote sensing imagery is a more economic data source compared with the LiDAR data and multiview imagery, its limited information results in great challenges and restricts the performance of existing monocular reconstruction methods. Moreover, the expensive cost and the limited quantity of 3-D annotations also restrict the application scenes of existing methods, which are mostly based on fully supervised learning. In our previous work, we have proposed MTBR-Net, a monocular building reconstruction method that consists of a fully supervised multitask network and a postprocessing module for optimizing the reconstruction results. In this work, we further propose WS-MTBR-Net, a weakly supervised building reconstruction network that uses fewer 3-D annotations and achieves better performance in an end-to-end manner. Specifically, our WS-MTBR-Net fully leverages the relationship between different components of a 3-D building instance and the property of off-nadir images to improve the footprint segmentation boundary, based on six modified tasks and a new network structure with an improved feature warping module to support weakly supervised learning. We also design a new training strategy via a hybrid loss function that enables using the training samples with different annotation levels, i.e., complete 3-D annotations, 2-D footprint annotations, and image-level angle annotations. The results on the BONAI Shanghai and Xi’an test datasets demonstrate that our method achieves competitive performance when using 50% fewer 3-D-annotated samples, and improves the footprint segmentation
$F1$
-score by around 4% compared with current state-of-the-art.