3-D Building Instance Extraction From High-Resolution Remote Sensing Images and DSM With an End-to-End Deep Neural Network

3-D building models play a vital role in numerous applications including urban planning and smart cities. Recent 3-D building modeling methods either rely heavily on available manually collected footprint references or hardly reach real automation on par with manual editing. To approach the automated extraction of instance-level 3-D buildings at level of detail 1 (LoD1), we introduce an innovative end-to-end 3-D building instance segmentation model. This model predicts accurate contours and heights of individual buildings simultaneously using ortho-rectified high-resolution remote sensing images and digital surface models (DSMs), getting rid of additional reference data and empirical parameter settings. First, we propose an anchor-free multihead (AFM) building extraction network tailored for extracting 2-D building contours. AFM incorporates a full-resolution, long-range correlation boosted global mask prediction branch along with anchor-free bounding box generation, as well as a newly developed online hard sample mining (OHSM) training procedure based on uncertainty analysis to emphasize error-prone positions in locating building contours. Subsequently, we incorporate a height prediction component to AFM in order to derive accurate building height information, thus creating the comprehensive 3-D building extraction model referred to as AFM-3D. The two-stage AFM-3D operates by initially predicting 3-D cube proposals, followed by generating refined 3-D prismatic models (LoD1 models) for each proposal. Thorough experimentation across different datasets demonstrates the superior performance of AFM and AFM-3D. A significant enhancement of 6.4% quality score is observed on the urban 3-D dataset in comparison to recent methods. In addition to the proposed novel methodology, we compare anchor-based and anchor-free bounding box generation mechanisms for remote sensing data, explore pixel-based and contour-based segmentation strategies, evaluate learning-based and empirical height estimation methods, and discuss the indispensability of DSM data in 3-D building instance extraction. These analyses yield valuable insights that contribute to the progression of 3-D building extraction research.