Omnidirectional Drift Control of an Underwater Biomimetic Vehicle-Manipulator System via Reinforcement Learning

This paper proposes an omnidirectional drift control on an underwater biomimetic vehicle-manipulator system (UB-VMS). The UBVMS has two biomimetic propellers, they obtain propulsive force by actuating their undulating fins. A configuration of the system, the dynamics of the UBVMS and the kinematics of the fins are given respectively. Then the control problem is designed as a Markov decision process (MDP). A reinforcement learning method based on the twin delayed deep deterministic policy gradient (TD3) is proposed for this MDP. A control policy is well trained by the reinforcement learning method and tested in eight different simulations. An analysis of the simulation results is also given.