An Adaptive Approach to Learn Overcomplete Dictionaries With Efficient Numbers of Elements

Dictionary learning for sparse representation has recently attracted attention among the signal processing society in a variety of applications such as denoising, classification, and compression. The number of elements in a learned dictionary is crucial since it governs specificity and optimality of sparse representation. Sparsity level, number of dictionary elements, and representation error are three correlated factors in which setting each pair of them results in a specific value of the third factor. An arbitrary selection of the number of dictionary elements affects either the sparsity level or/and the representation error. Despite recent advancements in training dictionaries, the number of dictionary elements is still heuristically set. To avoid the representation’s suboptimality, a systematic approach to adapt the elements’ number based on input datasets is essential. Some existing methods try to address this requirement such as enhanced K-SVD, sub-clustering K-SVD, and stage-wise K-SVD. However, it is not specified under which sparsity level and representation error criteria their learned dictionaries are size-optimized. We propose a new dictionary learning approach that automatically learns a dictionary with an efficient number of elements that provides both desired representation error and desired average sparsity level. In our proposed method, for any given representation error and average sparsity level, the number of elements in the learned dictionary varies based on content complexity of training datasets. The performance of the proposed method is demonstrated in image denoising. The proposed method is compared to state-of-the-art, and results confirm the superiority of the proposed approach.