Covariance-Assisted Matching Pursuit

We consider the problem of greedy sparse approximation in the presence of noise, given a-priori knowledge of the sparse coefficients' covariance and mean. The proposed Covariance-Assisted Matching Pursuit (CAMP) combines the a-priori knowledge by leveraging the Gauss-Markov theorem, and provides significantly better performance than the classical Orthogonal Matching Pursuit (OMP). This improvement is achieved by solving in each matching pursuit stage a weighted least-squares problem that provides the best linear unbiased estimator of the sparse representation. The covariance and mean of the sparse representation coefficients can be estimated by a simple procedure from training data, and the advantage of the proposed approach is demonstrated for the tasks of denoising and inpainting 150,000 patches from natural images.