Learning Sparse Representations of Depth

This paper introduces a new method for learning and inferring sparse representations of depth (disparity) maps. The proposed algorithm relaxes the usual assumption of the stationary noise model in sparse coding. This enables learning from data corrupted with spatially varying noise or uncertainty, such as that obtained by laser range scanners or structured light depth cameras. Sparse representations are learned from the Middlebury database disparity maps and then exploited in a two-layer graphical model for inferring depth from stereo, by including a sparsity prior on the learned features. Since they capture higher order dependencies in the depth structure, these priors can complement smoothness priors commonly used in depth inference based on Markov random field (MRF) models. Inference on the proposed graph is achieved using an alternating iterative optimization technique, where the first layer is solved using an existing MRF-based stereo matching algorithm, then held fixed as the second layer is solved using the proposed nonstationary sparse coding algorithm. This leads to a general method for improving solutions of state-of-the-art MRF-based depth estimation algorithms. Our experimental results first show that depth inference using learned representations leads to state-of-the-art denoising of depth maps obtained from laser range scanners and a time of flight camera. Furthermore, we show that adding sparse priors improves the results of depth estimation methods based on graph cut optimization of MRFs with first and second order priors.