Adaptive Image-Based Visual Servoing for Hovering Control of Quad-Rotor

Image-based visual servoing (IBVS) achieves precise positioning and motion control for a relatively stationary target by visual feedback, but problems persist with convergence and stability. Appropriate servoing gains for the IBVS are critical to the convergence and stability, but this control gain is heuristically a constant for most IBVS applications. This paper proposes an integrated method that allows adaptive adjustment of the servoing gain by reinforcement learning (RL) for IBVS control. The proposed method learns a policy to determine the value of the servoing gain on the fly. To ensure rapid convergence for the RL, truncating Q-learning (TQL) with faster convergence is used as learning algorithm, which uses truncated temporal differences (TDs) to update the TD. A nonuniform state space partitioning as a state encoder for RL allows more efficient policy. A strategy that uses the Metropolis derived from the simulated annealing is introduced for selecting the action, in order to balance exploration and exploitation so as to accelerate the learning speed. The integrated IBVS control system is tested using experiments involving a quad-rotor helicopter hovering control. The results of simulation and experiment show that the integrated IBVS method increases stability and ensures more rapid convergence than other methods.