Parameter-free Small Variance Asymptotics for Dictionary Learning

Learning redundant dictionaries for sparse representation from sets of patches has proven its efficiency in solving inverse problems. However, the optimization process often calls for the prior knowledge of the noise level or the regularization parameters for sparse encoding. In a Bayesian framework, these parameters are integrated within the probabilistic model through the choice of prior distributions. Although efficient, these methods come with numerical disadvantages for large-scale data. Small-variance asymptotic (SVA) approaches pave the way to much cheaper though approximate methods for inference by taking advantage from a fruitful interaction between Bayesian models and optimization algorithms. We propose such a SVA analysis of a Bayesian dictionary learning (DL) model where the noise level and regularization level are jointly estimated so that nearly no parameter tuning is needed. We analyze this algorithm and demonstrate its efficiency on real data to illustrate the relevance of the resulting dictionaries.