Path Following Control for Unmanned Surface Vehicles: A Reinforcement Learning-Based Method With Experimental Validation

In this article, a reinforcement learning (RL)-based strategy for unmanned surface vehicle (USV) path following control is developed. The proposed method learns integrated guidance and heading control policy, which directly maps the USV’s navigation states to motor control commands. By introducing a twin-critic design and an integral compensator to the conventional deep deterministic policy gradient (DDPG) algorithm, the tracking accuracy and robustness of the controller can be significantly improved. Moreover, a pretrained neural network-based USV model is built to help the learning algorithm efficiently deal with unknown nonlinear dynamics. The self-learning and path following capabilities of the proposed method were validated in both simulations and real sea experiments. The results show that our control policy can achieve better performance than a traditional cascade control policy and a DDPG-based control policy.