Optimal Impulsive Control for Discrete Stochastic Systems Through ADP

This paper constructs a novel general impulsive transition matrix in order to reveal the impulsive dynamics of the stochastic systems across the impulsive “events”. On the foundation of this matrix, the policy iteration based impulsive adaptive dynamic programming (IADP) algorithm and its more efficient version, are developed to obtain the optimal impulsive controller design scheme for stochastic systems. Monotonicity and optimality analysis are provided to show that both the IADP and the efficient IADP (EIADP) algorithms can find the global optimal performance index function. The proposed EIADP algorithm updates the iterative policies partially at each iteration step according to the actual hardware constraints, enabling the ADP-based algorithms to run on computing devices with limited memory spaces. Finally, a simulation experiment justifies the theoretical conjectures.