Global path planning for amphibious unmanned vehicles with multiple constraints via deep reinforcement learning

In this paper, a deep reinforcement learning (DRL) method for the global path planning of the water-air amphibious unmanned vehicle (WAAUV) is proposed. Firstly, the mathematical model of the WAAUV is constructed in line with the characteristics of an unmanned surface vehicle (USV) and a quadrotor unmanned aerial vehicle (UAV). Secondly, a Markov decision process (MDP) framework of the deep deterministic policy gradient (DDPG) algorithm is established to plan the path during navigation and flight, including the state space, the action space, and reward functions. In particular, the novel reward functions which overcome the sparse reward problem, are designed by fully considering the multiple constraints, such as reaching the goal point, energy consumption, obstacle avoidance, and so on. The control policy is trained with repeated trials by adjusting relevant parameters. Finally, through the simulations of the WAAUV in two modes, the feasibility and effectiveness of the proposed method is demonstrated.