Effective Probabilistic Neural Networks Model for Model-Based Reinforcement Learning USV

Gaussian process (GP) offers a robust solution for modeling the dynamics of unmanned surface vehicles (USV) in model-based reinforcement learning (MBRL). However, the rapidly increasing computational complexity with a large sample capacity of GP limits its application in complex scenarios that require substantial samples to cover the state space. In this article, a novel probabilistic MBRL approach, probabilistic neural networks model predictive control (PNMPC) is proposed to tackle this issue. With an iterative learning framework, PNMPC properly models the USV dynamics using neural networks from a probabilistic perspective to avoid the computational complexity associated with sample capacity. Employing this model to effectively propagate system uncertainties, a model predictive control (MPC) policy is developed to robustly control the USV against external disturbances. Evaluated by position-keeping and multiple targets-tracking scenarios on a real USV data-driven simulation, the proposed method consistently demonstrates its significant superiority in both model accuracy and control performance compared to not only GP model-based approaches but also the probabilistic neural networks-based MBRL baselines, across various scales of external disturbances.Note to Practitioners—Modelling the system dynamics and maintaining computational efficiency with a large sample set has been challenging for MBRL in the USV domain. We propose a novel neural network modeling method to capture the dynamic features of USV within an RL loop and develop a robust MPC policy based on its uncertainty propagation. Our method achieves computational complexity independent of the sample capacity and outperforms related baselines in model accuracy and control performance.