Reinforcement Learning Tracking Control for Unknown Continuous Dynamic Systems

Reinforcement learning tracking control (ILC) is proposed to solve the traditional iterative learning control (ILC) problem. For unknown continuous dynamic systems, precision output tracking is achieved for a given trajectory after several times of trials. The controller is composed of a feedback part and a feedforward part, where the feedback part uses linear states feedback to stabilize the system and the feedforward part is a time-dependent signal to ensure precision tracking. The desired trajectory is defined in a finite time interval and the controller is applied repeatedly to the system with the feedforward signal updated by reinforcement learning in each trial. The tracking problem is treated as a black-box optimization problem where the integral quadratic tracking error is a cost to be minimized under input constraints. In order to apply reinforcement learning, the feedforward signal is approximated by spline interpolation of a few representative points distributed uniformly along the tracking time interval. A search strategy based on squeeze theorem is adopted to update the value of the representative points to achieve minimum tracking error cost. The cart pole example illustrates the effectiveness of the proposed method.