Global and Local Awareness: Combine Reinforcement Learning and Model-Based Control for Collision Avoidance

In this research, we focus on developing an autonomous system for multiship collision avoidance. The proposed approach combines global path planning based on deep reinforcement learning (DRL) and local motion control to improve computational efficiency and alleviate the sensitivity to heading angle changes. To achieve this, firstly, DRL is used to learn a policy that maps observable states of target ships to a sequence of predicted waypoints. This learning task aims to generate a specific trajectory while avoiding collision with target ships complying with the international regulations for preventing collisions at sea (COLREGs). The learned policy is used as a global path planner during navigation. Secondly, the line-of-sight (LOS) guidance system is applied to calculate the desired course command based on the collision-free trajectory generated according to the policy. Lastly, a model-based control strategy is implemented to control the ship to the specific goal in collision-free space while satisfying the desired commands. We demonstrate the performance of the approach using an example of an autonomous surface vehicle. In comparison to other methods, our proposed control can provide a more stable and smoother maneuvering effect.