A New Algorithm for Dictionary Learning Based on Convex Approximation

The purpose of dictionary learning problem is to learn a dictionary D from a training data matrix Y such that Y ≈ DX and the coefficient matrix X is sparse. Many algorithms have been introduced to this aim, which minimize the representation error subject to a sparseness constraint on X. However, the dictionary learning problem is non-convex with respect to the pair (D,X). In a previous work [Sadeghi et at., 2013], a convex approximation to the non-convex term DX has been introduced which makes the whole DL problem convex. This approach can be almost applied to any existing DL algorithm and obtain better algorithms. In the current paper, it is shown that a simple modification on that approach significantly improves its performance, in terms of both accuracy and speed. Simulation results on synthetic dictionary recovery are provided to confirm this claim.