Depth Image Restoration for High-Level Vision Tasks

High-level computer vision tasks often necessitate large-scale, high-quality depth images. However, real-world depth images are unsuitable for direct application in experimental studies due to their inherent noise and presence of holes. Consequently, image restoration becomes imperative. Large-scale hole repair often proves ineffective in the face of realistic and complex degraded scenes. To address this issue, our research introduces the fast Fourier convolution and a Multi-Scale Attention mechanism into an unsupervised learning network. This integration expands the network's sensory field and enhances perception of spatial feature details within the depth image. Its objective is to improve the restoration performance of depth images that possess substantial regions of missing depth values. Our experimental results demonstrate significant enhancements achieved by our restoration algorithm. Specifically, relative to the original depth images, the algorithm improves PSNR and SSIM by 25% and 60%. On average, improvements exceed the baseline by 9% and 15%, respectively. Moreover, our algorithm outperforms the original depth images in target detection, with the average precision values for human pose recognition tasks increasing by 24% and 25%, respectively. Compared to the baseline, these AP values experience additional enhancements of 11% and 18%, respectively. The restoration algorithm presented in this paper not only achieves visually pleasing results in terms of image quality but also significantly enhances the performance of downstream advanced vision tasks.