Autonomous robot navigation with self-learning for collision avoidance with randomly moving obstacles

This paper develops a hierarchical controller to avoid randomly moving obstacles in autonomous navigation of a robot. The developed method consists of two parts: a high-level Q-learning controller for choosing an optimal plan for navigation and a low-level, appearance-based visual servo (ABVS) controller for motion execution. The use of robot learning ability in collision avoidance is a novel feature, in a combined system framework of planning and visual servo control. The developed approach takes advantage of the on-board camera of robot whose finite field of view is naturally suitable for the Q-learning algorithm. Because of the Q-learning controller, knowledge of obstacle movement and a control law for the ABVS controller are not needed. This is a significant computational advantage. The method is implemented in a simulation system of robot navigation. The results show that Q-learning, which is a method of reinforcement learning, successfully converges to an optimal strategy for the robot to establish a proper motion plan.