Reinforcement Learning-Based Decentralized Control for Networked Interconnected Systems With Communication and Control Constraints

This paper investigates the optimal decentralized control issue for a class of networked interconnected systems (NISs) under communication and control constraints. First of all, the decentralized optimal control problem for NISs is transformed into the optimal control issue for nominal systems. And then the event-triggered Hamilton-Jacobi-Isaac (HJI) equations are derived with optimal control theory based on the constructed cost function and adopted adaptive event-triggered mechanism (AETM). Furthermore, a Reinforcement Learning (RL) based event-triggered optimal control algorithm with actor-critic networks is proposed to obtain numerical solutions of the HJI equations and the Uniformly Ultimately Bounded (UUB) stability of NISs is proved to be guaranteed with Lyapunov stability theorem. Eventually, a simulation experiment is conducted to verify the effectiveness of proposed RL-based optimal decentralized algorithm with AETM. Note to Practitioners—Actually, the limitation of communication resource is one of the key factors leading to performance degradation of NISs. It is crucial to cope with this issue in the control of NISs. Employing proper communication protocol is an effective way to diminish the adverse effects of undesirable phenomena, such as data loss and out-of-order. In addition, given the scale is gradually expanded and the structure is gradually complex for NISs, the operation cost is a problem that cannot be ignored in the control process of NISs. Thus, it is significant and challenging to minimize the operation cost on the basis of ensuring system stability. In view of the communication constraints and optimization problem, the HJI equations based on AETM are derived to model the decentralized optimal control problem for NISs. Moreover, a RL-based iterative algorithm is proposed to derive the numerical solutions to HJI equations and further effectively stabilize the targeted control constrained NISs in the sense of UUB.