Design and Field Test of Collision Avoidance Method With Prediction for USVs: A Deep Deterministic Policy Gradient Approach

Autonomous collision avoidance technology is the core of unmanned surface vehicles (USVs). Deep reinforcement learning (DRL) is a new approach to avoid collision for USVs. However, most research is based on the assumption of a fixed number of obstacles and ignores the collision prediction to improve safety. To address this problem, a novel “prediction-decision” collision avoidance model based on the deep deterministic policy gradient (DDPG) is proposed. First, a radiation-shaped state space is designed to make the DDPG that can be used in time-varying scenarios with stochastic obstacles. Then, the velocity obstacle (VO) is combined with the state space for training to realize the collision prediction. Subsequently, reward functions are designed using a reward-shaping technique to improve training efficiency and safety. Finally, virtual simulation experiments based on Unity3D and field tests are conducted to verify the algorithm’s performance. The results show that it can take safe collision avoidance actions in unknown environments and with generalization ability.