Hierarchical optimal synchronization for heterogeneous nonlinear dynamical network systems with input saturation via reinforcement learning

This paper studies the hierarchical optimal synchronization(HOS) problem of heterogeneous nonlinear complex dynamic networks with input saturation constraints under the sequential decision-making mechanism. Firstly, a Stackelberg-Nash game model to solve the HOS problem is established, and the value functions of the major node and the minor nodes are introduced respectively. Then, the coupled HJB equations are derived based on the controlled network models. Furthermore, the expressions of optimal controllers for the major node and the minor nodes are theoretically given. Integral reinforcement learning algorithm based on value iteration method is designed, and the convergence, optimality, admissibility of iterative policy of the algorithm are analyzed. To implement this algorithm, the single-critic neural networks(NNs) are used to estimate the value functions, the gradient descent technique is employed to train NNs to toward the ideal weights. Finally, numerical simulations are provided to check the validity of the main results.