Underactuated Navigation Actor-Critic Deep Reinforcement Learning Framework for Holistic Path Planning of Uncrewed Surface Vehicles

If underactuated dynamics can not be accommodated in path planning for an uncrewed surface vehicle (USV), sway unactuation makes the path untrackable, thereby threatening navigation resilience. In this paper, an underactuated navigation actor-critic (UNAC) deep reinforcement learning (DRL) framework is devoted to feasibly trackable path planner for an underactuated USV. By integrating a long short-term memory module into the critic network, historical state sequences are compressed into low-dimensional representations, thereby balancing optimization efficiency and complexity. To incrementally optimize pertinent path, a composite reward function covering process, collision and target approaching is created to fertilize the optimizer. Within the algorithmic flow, successive waypoints-tracking mechanism is embedded, ensuring that path-planning policy can be compatible with unactuated sway dynamics. To provide sufficiently diversified learning scenarios that can hardly experience in practice, Unity3D-based virtual-reality environments are established by replicating real-world shallow and congested situations, showcasing that the UNAC-based path planner works resiliently under unfamiliar circumstances. Compared to conventional DRL methods, the UNAC-DRL framework not only accelerates the learning process but also achieves a success rate improvement of up to 15%.