DRL-Based Motion Control for Unmanned Surface Vehicles with Environmental Disturbances

This paper focuses on the motion planning problem of unmanned surface vehicles (USVs) under wind and wave conditions. Environmental dynamic factors such as wind and waves can cause disturbances to the motion of USVs during navigation. To address this issue, the twin delayed deep deterministic policy gradient (TD3) algorithm based on deep reinforcement learning (DRL) is introduced to combine a low-level proportional-integral (PI) controller. Then, a TD3-PI approach is proposed to adjust the PI parameters in complex environments so as to cope with wind and wave disturbances. A three-dimensional simulation environment is constructed to train the neural networks of the proposed approach. After the training process, heading angle experiments and trajectory tracking experiments under different wind and wave conditions are set up to evaluate the stability and capability of the PI algorithm and our proposed TD3-PI approach. The result shows that the TD3-PI approach outperforms the PI algorithm in terms of stability and efficiency under wind and wave conditions.