Thresholding RULES and iterative shrinkage/thresholding algorithm: A convergence study

Imaging inverse problems can be formulated as an optimization problem and solved thanks to algorithms such as forward-backward or ISTA (Iterative Shrinkage/Thresholding Algorithm) for which non smooth functionals with sparsity constraints can be minimized efficiently. However, the soft thresholding operator involved in this algorithm leads to a biased estimation of large coefficients. That is why a step allowing to reduce this bias is introduced in practice. Indeed, in the statistical community, a large variety of thresholding operators have been studied to avoid the biased estimation of large coefficients; for instance, the non negative Garrote or the the SCAD thresholding. One can associate a non convex penalty to these operators. We study the convergence properties of ISTA, possibly relaxed, with any thresholding rule and show that they correspond to a semi-convex penalty. The effectiveness of this approach is illustrated on image inverse problems.