AGNet: Attention Guided Sparse Depth Completion Using Convolutional Neural Networks

Sparse depth completion generates a dense depth image from its sparse measurement with the guidance of RGB image. In this paper, we propose attention guided sparse depth completion using convolutional neural networks, called AGNet. We adopt attention learning to get geometric cues for depth regression from RGB image and capture multi-scale depth structures. First, we use RGB image and valid binary mask from the input sparse depth image as input to generate an initial coarse depth image and its confidence map. Then, we generate attention map for depth refinement using a cross spatial attention module (CSAM). CSAM separately takes the RGB image with valid mask and invalid mask as input to make full use of color information in attention map. Next, we build a multi-scale learning network to encode the sparse depth image with different scales, thus leading to accurate depth completion. AGNet takes advantage of the input sparse depth image for encoding coarse features with a moderate model size. Experimental results show that AGNet achieves comparable performance with state-of-the-art methods for depth completion in NYU v2 dataset.