An Autonomous Decision-making Algorithm for Ship Collision Avoidance Based on DDQN with Prioritized Experience Replay

In order to reduce the influence of human factors on ship collision avoidance, this paper proposes an autonomous collision avoidance decision-making algorithm based on Double Deep Q Network (DDQN) with prioritized experience replay (PER). Based on a full understanding of the International Regulations for Preventing Collision at Sea (COLREGs), and the reward function of the autonomous collision avoidance decision-making algorithm is guided and designed in terms of the number of rudder operations, the amount of the cumulative steering angle during the avoidance process, the DCPA when clear of the other ship, the deviation distance at the end of the avoidance, the total magnitude of ship heading changes during the avoidance process and rule compliance by combining with the statistical results of real water traffic data. Ship encounter scenarios are constructed based on COLREGs, which are arranged and combined the sample training and testing sets for the agent; We compare the algorithm proposed in this paper with DQN, DQN with prioritized experience replay and DDQN algorithms by simulating the same encounter scenario, using this as the training set for the model; Finally, multi-ship encounter scenario simulations are set up as the testing set for the model to verify the performance of the algorithm. The results show that compared with three other reinforcement learning algorithms, DDQN with prioritized experience replay algorithm can make ship collision avoidance decision-making more efficient and improve the safety of ship autonomous navigation under COLREGs; and the trained model can be replicated and applied to a system with any number of agents to provide a reference for ship autonomous collision avoidance research.