Application of Reinforcement Learning to autonomous heading control for bionic underwater robots

The bionic underwater robot propelled by undulating fins is an interesting field in current research on underwater robots. With the prosperous development of bionic underwater robots, its control problem remains big challenging for strong nonlinearity, uncertainty environments, and lack of understanding of dynamic characteristics of undulating fins. As a model-free method, the Q-learning based reinforcement learning achieves its control motivation by interacting with the environment and maximizing a reward, so suits the complicated applications such as robot control. This paper introduced the online Q_learning algorithm to the autonomous heading control for a kind of bionic underwater robot with two undulating fins. The algorithm doesn't need to know any knowledge about the robot, and can learn the internal mapping between states and actions that control behaviors must contain. With the simulation experiments, the validity of reinforcement learning algorithm in autonomous heading control of the bionic underwater robot was validated.