LaSalle-Type Theorem and Its Applications to Infinite Horizon Optimal Control of Discrete-Time Nonlinear Stochastic Systems

Based on discrete martingale theory, the LaSalle-type theorem for general discrete-time stochastic systems is obtained and the almost sure stability is in turn discussed. As applications, infinite horizon nonlinear optimal regulator is investigated, and a dynamical programming equation called the Hamilton-Jacobi-Bellman equation is also derived for discrete-time nonlinear stochastic optimal control.