Q-Learning for Stackelberg Games of Uncertain Nonlinear Systems

The Stackelberg equilibrium approach is based on the traditional dynamic programming technique, which requires an understanding of the system model to solve the optimal control problem. Therefore, we propose a Q-learning algorithm based on actor-critic structure to solve the optimization problem of Stackelberg game in nonlinear systems with unknown system model data. Currently, the Q-learning algorithm is mainly used in the study of linear systems in this direction, and the study of nonlinear system problems is limited. In addition, the stability of the system is ensured by proving the consistency of the system state and the ultimate boundedness of the neural network weight estimation error. Finally, the effectiveness of the algorithm is verified by simulation experiment.