An Improved Deep Reinforcement Learning Method-Based Nonparametric Modeling of Ship Dynamics

For the nonparametric modeling problem of ship maneuvering motion, this paper proposes a ship dynamics modeling method benefiting from the prioritized experience replay (PER) and deep deterministic policy gradient (DDPG) algorithm, so-called PER-DDPG. Because experience replay mechanism of DDPG has the problem of low sample utilization, this paper introduces PER to speed up the learning efficiency and improve the strategy quality of the agent. In the modeling process, taking Mariner ship as the research object, the state space, action space of the algorithm and the reward function aiming at minimizing the error of ship motion variables are designed. In order to verify the effectiveness of PER-DDPG algorithm, the algorithm is compared with DDPG algorithm in five groups of zigzag maneuver experiments under different rudder angles. The results show that the identification model based on PER-DDPG has higher prediction accuracy and better generalization ability. The root mean square error and determination coefficient of model on the validation dataset are 0.0011∼0.0033 and 0.9784∼0.9986, respectively.