A Model-Free Optimal Control Method

In this article, we propose a general model-free optimal control method. The control input is first parameterized using some basis functions. Then the parameters are updated using gradient descent where the gradient of the cost function with respect to the parameters is obtained by finite difference. This seems to mimic the idea of reinforcement learning but the difference lies in that the proposed algorithm is an open-loop control. The convergence of this algorithm is proved. The algorithm is tested in (i) a continuous system; (ii) the multiple-region state-dependent switched system without prior information on the switching sequences; and (iii) a time-delay system. The results are compared with the respective exact solution or existing algorithm. Simulation results show that the performance is as well as the optimal solution.