Stochastic Fista Algorithms: So Fast ?

Motivated by challenges in Computational Statistics such as Penalized Maximum Likelihood inference in statistical models with intractable likelihoods, we analyze the convergence of a stochastic perturbation of the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA), when the stochastic approximation relies on a biased Monte Carlo estimation as it happens when the points are drawn from a Markov chain Monte Carlo (MCMC) sampler. We first motivate this general framework and then show a convergence result for the perturbed FISTA algorithm. We discuss the convergence rate of this algorithm and the computational cost of the Monte Carlo approximation to reach a given precision. Finally, through a numerical example, we explore new directions for a better understanding of these Proximal-Gradient based stochastic optimization algorithms.