Online optimal auto-tuning of PID controllers for tracking in a special class of linear systems

This paper proposes a reinforcement learning (RL) algorithm based on approximate dynamic programming to optimally auto-tune a Proportional Integral Derivative (PID) controller by solving an infinite-horizon optimal tracking control problem for a special class of linear systems. The algorithm is based on an actor/critic framework where a critic approximator is used to learn the optimal cost and an actor approximator is used to learn the optimal PID gains. The adaptive control nature of the algorithm requires a persistence of excitation condition to be a-priori validated, but this can be relaxed by using previously stored data concurrently with current data in the tuning of the critic approximator. Simulation results show the effectiveness of the proposed approach for a stirred-tank plant reactor.