Research on D3QN-PER Path Planning Method Based on LSTM-GRU Composite Model

Reinforcement learning plays a pivotal role in Autonomous Underwater Vehicle (AUV) path planning by enabling the AUV to learn optimal navigation strategies through trial and error. However, traditional Deep Q Network (DQN) suffers from slow convergence and instability when applied to AUV path planning. In this paper, a Prioritized Experience Replay Dueling Double Deep Q-learning method based on the Long Short- Term Memory-Gate Recurrent Unit composite model (LSTM-GRU-D3QN-PER) is proposed to address this issue. Firstly, the Deep Double Q Network (D3QN) is used to mitigate the overestimation problem in traditional Q-learning algorithms and to improve the estimation accuracy of the value function, thus improving the stability and convergence of training. Secondly, the prioritized experience replay mechanism is employed in the method to enhance the sample efficiency of training and improve the convergence speed and stability of the algorithm. Finally, the Long Short- Term Memory-Gate Recurrent Unit (LSTM-GRU) composite model is employed to handle state information, resulting in the improvement of memory and cognitive abilities to obtain more favorable environmental information. To demonstrate the efficacy of the proposed algorithm, experiment is executed on the Open AI Gym testing platform and validated in the Maze environment. Experiment results show that the algorithm with potential to improve convergence speed, stability, and learning performance.