Improved dictionary learning by constrained re-training over residual components

The dictionary learning process aims at training for a dictionary that can loyally and sparsely represent data in a given training set. In this paper, we propose performing a second pass of dictionary learning where the training set is composed of the residuals of the original training set as calculated with respect to the outcome of a first pass of dictionary learning. In the second pass, the dictionary is updated with the residual signals. However, the representation fidelity of the original training set is imposed. This is formulated as a constrained optimization problem and is solved using Lagrange multipliers with a line-search. The proposed strategy is shown to train dictionaries with better representation capabilities compared to dictionaries trained with standard dictionary learning. This result is validated with tests concluded over the problem of image representation.