Optimal Tracking Control for Cyber-Physical Systems Under Mixed Attacks via Game-Theoretical Q-Learning

This paper investigates the optimal tracking control problem for cyber-physical systems (CPSs) under mixed attacks based on the Stackelberg game strategy. First, an improved value function is designed to meet performance criteria by considering the control signal, mixed attack signal, and tracking performance. Then, based on the Stackelberg game theory and the principle of optimality, the optimal control strategy and false data injection attack policy are derived by solving a coupled algebraic Riccati equation (CARE). The proposed control strategy can effectively alleviate the adverse impact of mixed attacks on the control performance of CPSs. Subsequently, sufficient conditions are provided to guarantee the existence of the solution to the CARE. Additionally, an improved Q-learning algorithm is proposed to learn the optimal control scheme through state reconstruction, which avoids the need for access to state vectors and facilitates data-based controller design. Using the Lyapunov stability theory, it is demonstrated that the presented algorithms are convergent and the output of CPSs can track the reference trajectory. Finally, the proposed approach is validated by numerical simulations. Note to Practitioners—In the engineering application scenarios, the sharing feature of network communication may expose the controlled system to malicious attacks (such as DoS attacks and FDI attacks). The majority of the current control methods focus on one-sided analyses. In this paper, the dynamic interaction between attackers and defenders is described using a Stackelberg game model. Within this model, an optimal tracking control algorithm is proposed to mitigate the impact of mixed attacks on CPSs. Moreover, sufficient conditions for tolerable probability of attack are derived, which enables practitioners to determine the conditions under which the attacked system’s stable tracking performance may still be maintained.