Dynamic Obstacle Avoidance Method for Unmanned Surface Vessel Based on Deep Reinforcement Learning and Dynamic Window Approach

The obstacle avoidance technology of Unmanned Surface Vessels (USV) is the key to ensuring their safe navigation. This article proposes a collision avoidance technology for USV based on Deep Deterministic Policy Gradient (DDPG) and Dynamic Window Approach (DWA), aiming to solve the obstacle avoidance problem of USV. Based on the idea of DWA, the action space of DDPG was designed. A new reward mechanism is proposed according to the trajectory evaluation function standards in the DWA algorithm, which solves the sparse reward problem of traditional DDPG and improves the convergence and obstacle avoidance effects of the algorithm. To demonstrate the effectiveness of the algorithm, a multi-ship collision avoidance experiment was designed. The results indicate that the proposed algorithm can achieve effective collision avoidance.