Adaptive Reinforcement Learning for Fault-Tolerant Optimal Consensus Control of Nonlinear Canonical Multiagent Systems With Actuator Loss of Effectiveness

This article addresses the adaptive optimized consensus tracking control problem of nonlinear multiagent systems (MASs) via a reinforcement learning (RL) algorithm. Specifically, the nonlinear high-order MASs are formulated in a canonical form, with considerations for both actuator effectiveness loss and time-varying bias faults. First, neural networks (NNs) are utilized to approximate unknown nonlinear dynamics, and a state identifier and a fault estimator based on NNs are established, both of which are essential for evaluating state information and bias faults, respectively. Second, to achieve a high-order canonical dynamic consensus and enhance the efficiency of the consensus control strategy, a sliding-mode mechanism is employed to regulate tracking errors. Moreover, we develop an adaptive NN-based fault-tolerant optimal control method by integrating the sliding-mode mechanism with an actor–critic structured RL algorithm. It is proved that the outputs of the MASs precisely align with the desired reference signals, while ensuring the boundedness of all closed-loop signals. Finally, the proposed control methodology's effectiveness is validated through a simulation example.

I. Introduction

During the past several decades, multiagent systems (MASs) have emerged as a focal point in control theory research, owing notably to their integral component in applications, such as drone fleets [1], autonomous cars [2], and smart grids [3]. Among the various control strategies for MASs [4], [5], consensus control has garnered considerable attention over the past decade. This method ensures coordinated and collaborative behavior in MASs, which facilitates the convergence of the state or output of follower agents with that of leader agents through local interactions. To provide a few, global consensus tracking control of MASs with prescribed performance was explored in Li et al.’s [6] work, and Wang et al. [4] investigated an adaptive bipartite consensus tracking control scheme for MASs. Moreover, the complexity of the operating environments renders MASs susceptible to unmeasurable system states and unknown external disturbances. To counteract these effects, state observers [7] and disturbance observers [8] were incorporated into adaptive consensus control strategies. However, a significant gap in the existing research is the lack of consideration for actuator faults, which are critical for the reliable operation of MASs.