Underwater Dynamic Tracking Control of AUV Based on Complex Environment Simulation and ACL-SAC Deep Reinforcement Learning

The current underwater tracking control methods based on optical image priori data have the problems of low reliability, high training cost, and poor adaptability to different environments. This paper proposes an underwater dynamic tracking control method based on complex environment simulation and deep reinforcement learning, which can solve out the optimal real-time yaw angle and propelling force of Autonomous Underwater Vehicle (AUV) for target tracking. Firstly, we design an underwater simulation environment generator which supports parallel training. This generator is able to provide a large number of complex stochastic environments containing random current effects and sound speed uncertainty for the deep reinforcement learning training. Then, a control algorithm based on Attention Conv-LSTM (ACL) and Soft Actor-Critic (SAC) networks is proposed, which has fast convergence speed and adaptation capability to complex environment. In addition, we design a composite reward function to avoid premature local optimal solutions. Experimental results show that the tracking detachment and collision probabilities under the simulation environment with random ocean currents and acoustic velocity uncertainty interference are as low as 0.12% and 0.82%, respectively. The AUV testing under real environment presents good dynamic tracking performance both for linearly and randomly moving targets.