Discounted Stable Adaptive Critic Design for Zero-Sum Games With Application Verifications

In this paper, an adaptive critic design with performance guarantee is established based on the discounted value iteration algorithm to settle with the optimal regulation problem for discrete-time zero-sum games. Value iteration is implemented to obtain the approximate optimal solutions to the Hamilton-Jacobi-Isaacs equation for nonlinear systems and the game algebraic Riccati equation for linear systems. Then, we focus on system stability affected by the introduction of the discount factor and the admissibility of the policy pairs in the value iteration process. The appropriate selection range of the discount factor and the criteria for ensuring system stability are established to assist in obtaining the stabilized optimal policy pair, which not only makes the cost function converge to the optimal value, but also guarantees the asymptotic stability of the closed-loop system. Finally, practical examples for the power system and the ball-beam system are conducted to demonstrate the effectiveness of the presented method. Note to Practitioners—Since there exist a multitude of dynamic systems with uncertainty and interference, the zero-sum game problems are ubiquitous, especially when dealing with dynamic systems featuring antagonistic properties. As an important research direction in the field of optimal control, zero-sum games usually involve designing policy pairs that can optimize the system performance in the presence of adversarial disturbances. Due to the excellent adaptability, value iteration in adaptive dynamic programming is employed to deal with this kind of issues. In addition to focusing on the optimality of policies, the system stability during the control process is equally significance, where the stability is the premise of all operations. Therefore, we are dedicated to providing guidance on the optimal regulation of discrete-time zero-sum games with performance guarantee, which contributes to obtain the stable optimal policy pair. Theoretical analysis of the stability is provided and the asymptotic stability of the system is ensured, which improves the performance of the designed controller. Furthermore, simulation experiments for practical applications are conducted, which verify the feasibility and effectiveness of the proposed control design.