Global Optimization for Stochastic Programming Via Sequential Monte Carlo Sampling

This paper proposes a continuous-time optimization algorithm for stochastic programming. We introduce the sequential Monte Carlo sampling scheme to do with the high dimensionality of the stochastic disturbance and the switching gradient Lagevin dynamics to reduce the computational requirement on evaluation of the gradient on large data set. Our algorithm is in continuous-time and can seek the global minimizer of stochastic programming problems. We fit our analysis into the framework of two-time-scale systems and propose a stochastic averaging approach to analyze the convergence of our optimization algorithm. With the help of Dirichlet form, Fredholm alternative and Poincare inequality and by means of the tool of Fokker-Planck equation, we provide a rigorous theoretical foundation for the analysis and design of the global optimization algorithm. The importance of our algorithm on the one hand stems from serving the application-oriented purpose such as dealing with high-dimensionality and reducing computational complexity, and on the other hand originates in providing some theoretically interesting techniques which is of direct applicability for global optimization problems.