Reinforcement Learning for Image-Based Visual Servo Control

In this paper, a continuous-time reinforcement learning (RL)-based controller is developed for image-based visual servoing (IBVS). The IBVS control dynamics is of the form where the drift term is absent and there is an uncertainty in the Jacobian matrix that is multiplied with the input. This poses a challenge for developing a continuous-time RL controller. The paper presents an actor-critic or synchronous policy iteration (PI)-based RL controller along with a parameter update law for the unknown parameter in the image Jacobian and proves closed-loop stability with the proposed controller. An infinite-horizon value function minimization objective is achieved by regulating the current image features to the desired with near-optimal control efforts. The proposed controller is tested using a simulation use case and the results validate the proposed theory.