Online Dictionary Learning with Confidence

Online dictionary learning has received intensive attention in signal processing field with streaming or dynamic data. Different from classical online dictionary learning methods that treat all atoms equally, in this paper, we present a novel online dictionary learning with a confidence parameter introduced on each of atoms. The confidence indicates the scale of the update of atoms during online learning; frequently estimated atoms are thus supposed to be updated less aggressively than those of low usage frequency. This updating mechanism is beneficial for learning with dirty examples. As a result, the outliers would be prevented from influencing much on the frequently used atoms that have been well estimated through the clean examples. In detail, we employ variance of the atoms to measure the confidence on the quality of the learned atoms. And frequently-used atoms are supposed to have smaller variances (i.e. more confidence) than those of low usage frequency. Our algorithm consists of two main parts, namely, confidence-weighted sparse coding and dictionary update with confidence fine-tuning, each of which can be solved efficiently by our designed optimization methods. In addition, due to the introduced confidence, our algorithm does not have to depend on the widely-used ℓ
₁
norm in order to realize the robustness against the outliers, which is usually computationally expensive in the training cost. Experimental results on synthetic and benchmark datasets demonstrate the imposed confidence on atoms in the dictionary can improve the performance of the learned dictionary. The quality of the obtained dictionary is comparable to that of state-of-the-art robust online dictionary learning methods, however, our method is much faster than those ℓ
₁
norm based ones.