Adaptive Optimal Control of Completely Unknown Systems with Relaxed PE Conditions

This paper proposes a novel identifier-critic (IC) learning control strategy for completely unknown nonlinear system. Different from the existing results, the proposed IC control is capable of obtaining the optimal control under relaxed persistence of excitation (PE). A neural network (NN) based identifier is established to approximate the unknown system dynamics. After that, an only-critic NN framework is proposed to solve the Hamiltonian-Jacobi-Bellman (HJB) equation such that the control policy is obtained. To estimate the unknown weights of both identifier NN and critic NN simultaneously without strictly PE limitation, the dynamic regressor extension and mixing (DREM) technique is introduced to design the NN weight update laws. Meanwhile, new easy-to-check online convergence conditions for the proposed adaptive laws are given to ensure the unknown weights converge to their ideal values. In addition, theoretical analysis is also given to prove the significant relaxation of the proposed convergence conditions compared with the standard PE assumption.