Height Prediction and Refinement From Aerial Images With Semantic and Geometric Guidance

Deep learning provides a powerful new approach to many computer vision tasks. Height prediction from aerial images is one of those tasks which benefited greatly from the deployment of deep learning, thus replacing traditional multi-view geometry techniques. This manuscript proposes a two-stage approach to solve this task, where the first stage is a multi-task neural network whose main branch is used to predict the height map resulting from a single RGB aerial input image, while being augmented with semantic and geometric information from two additional branches. The second stage is a refinement step, where a denoising autoencoder is used to correct some errors in the first stage prediction results, producing a more accurate height map. Experiments on two publicly available datasets show that the proposed method is able to outperform state-of-the-art computer vision based and deep learning-based height prediction methods. Code is publicly available at:
https://github.com/melhousni/DSMNet
.