Learning Hierarchical Color Guidance for Depth Map Super-Resolution

The color information are the most commonly used prior knowledge for depth map super-resolution (DSR), which can provide high-frequency boundary guidance for detail restoration. However, its role and functionality in DSR have not been fully developed. In this article, we rethink the utilization of color information and propose a hierarchical color guidance network (HCGNet) to achieve DSR. On the one hand, the low-level detail embedding (LDE) module is designed to supplement high-frequency color information of depth features in a residual mask manner at the low-level stages. On the other hand, the high-level abstract guidance (HAG) module is proposed to maintain semantic consistency in the reconstruction process by using a semantic mask that encodes the global guidance information. The color information of these 2-D plays a role in the front and back ends of the attention-based feature projection (AFP) module in a more comprehensive form. Simultaneously, the AFP module integrates the multiscale content enhancement (MCE) block and adaptive attention projection (AAP) block to make full use of multiscale information and adaptively project critical restoration information in an attention manner for DSR. Compared with the state-of-the-art methods on four benchmark datasets, our method achieves more competitive performance both qualitatively and quantitatively. The code and results can be found from the link of https://rmcong.github.io/HCGNet_TIM2024.