Two-Stage Enhancement Network for Monocular Depth Estimation of Indoor Scenes

Benefiting from convolutional neural networks (CNNs), monocular depth estimation has achieved remarkable performance. However, most CNNs focus on learning local details of the scene, resulting in texture deviation and further causing the misrepresentation of scene structure. In this paper, we propose a novel Two-Stage Enhancement Network (TSENet) for monocular depth estimation with indoor scenes, which can enhance the description of the structure and suppress layout outliers caused by texture deviation. Specifically, for the first stage, a multi-modal fusion module based on RGB-Depth (RGB-D) is proposed to enhance structural information by pasting depth regions into original images. For the second stage, we design a multi-scale feature correlation enhancement (MFCE) module to optimize global information by learning the correlation between global and local features. The correlation is obtained from the fusion of multiple pooling and convolutional layers, which can alleviate structural misrepresentation in decoding. Experiments and results on the challenging datasets NYUv2 and SUN RGB-D demonstrate that our TSENet achieves competitive performance compared with the state-of-the-art approaches.