Path-following control of underactuated ships using actor-critic reinforcement learning with MLP neural networks

In this paper, the critic-actor reinforcement learning method is proposed to solve the path-following problem of underactuated ships. In order to facilitate the design of the reward function, ship path-following control is transformed into calculating a reference course to reduce cross tracking error by the methods of coordinate compression and drift angle compensation. For the purpose of overcoming the curse of dimensionality caused by the continuous state spaces and action spaces of ships, the multilayer perception (MLP) neural networks are used to approximate the value function. Finally, the simulation with the environmental disturbances including wind, wave and ocean current, is presented to demonstrate the effectiveness and the performance of the proposed scheme.