Sparse Representation Based on the Analysis Model With Optimization on the Stiefel Manifold

Sparse representation with an analysis model becomes a challenging task because the optimization problem involves additional constraints on the analysis dictionary. We present the problem of learning the analysis dictionary through an optimization function with an orthogonality constraint, which can be generally solved by alternately implementing the analysis dictionary update and the sparse coding steps. In the dictionary update step, the traditional method for updating the analysis dictionary with the orthogonality constraint is to first perform the update in Euclidean space without taking the constraint into consideration and then project the solution onto a manifold where the orthogonality constraint is satisfied. However, this projection method is approximate, and the learned dictionary obtained by projection may not embody the inherent structure of the original dictionary. Thus, we develop a novel framework for seamlessly learning the analysis dictionary along with the manifold. This method can provide a more accurate and effective analysis dictionary. In the sparse coding problem, to avoid adjustment of the parameters of the formulation, we introduce an indicator function as the penalty function for optimization of the cost function. Thus, we reduce two parameters to one, which is easier to optimize. Then, we adopt a Douglas–Rachford-like scheme to solve the problem. From the results of the numerical experiments conducted to evaluate the recovery rate of the analysis dictionary, we can intuitively see the performance of the proposed algorithms. Furthermore, the proposed algorithms show good performance in image denoising for realistic applications.