Sparsity-based depth image restoration using surface priors and RGB-D correlations

In this paper we propose a sparsity-based, directional variational approach for upsampling depth images, aided by an accompanying optical (in RGB) image of higher spatial resolution. Compared to previously published works on RGB-D superresolution, the main innovations of this work are: 1. performing depth image restoration in an overcomplete sparsity space derived from the directionalities of the RGB image; 2. refining the regularization term of the underlying inverse problem by a cross-validating spatial discontinuities in the optical and depth images. By integrating these new techniques the proposed depth image superresolution method delivers very competitive performance against existing ones.