Dynamic Leader–Follower Output Containment Control of Heterogeneous Multiagent Systems Using Reinforcement Learning

This article addresses the optimal containment problem of heterogeneous multiagent systems (MASs) with dynamic leaders via reinforcement learning (RL), where the dynamics of all agents are all completely unknown. A distributed model-free observer is constructed for each follower to estimate the leaders’ dynamics and the output trajectories inside the convex hull formed by the leaders. Based on the designed observers, the optimal containment problem is formulated as an optimal tracking control issue. Then the discounted performance functions are introduced to obtain algebraic Riccati equations (AREs). And a model-free RL algorithm is developed to learn the AREs online. To implement this algorithm, we design a single critic neural network structure for each follower to approximate Q-function, and estimate optimal control policy and worst-case adversarial input policy. Finally, a numerical simulation is provided to demonstrate the effectiveness of the proposed algorithm.