Differential Game-Based Deep Reinforcement Learning in Underwater Target Hunting Task

To meet requirements for real-time trajectory scheduling and distributed coordination, underwater target hunting task is challenging in terms of turbulent ocean environments and dynamic adversarial environment. Despite the existing research in game-based target hunting area, few approaches have considered dynamic environmental factors, such as sea currents, winds, and communication delay. In this article, we focus on a target hunting system consisted of multiple unmanned underwater vehicles (UUVs) and a target with high maneuverability. Besides, differential game theory is leveraged to analyze adversarial behaviors between hunters and the escapee. However, it is intractable that UUVs have to deploy an adaptive scheme to guarantee the consistency and avoid the escape of the target without collision. Therefore, we conceive the Hamiltonian function with Leibniz’s formula to obtain feedback control policies. In addition, it proves that the target hunting system is asymptotically stable in the mean, and the system can satisfy Nash equilibrium relying on the proposed control policies. Furthermore, we design a modified multiagent reinforcement learning (MARL) to facilitate the underwater target hunting task under the constraints of energetic flows and acoustic propagation delay. Simulation results show that the proposed scheme is superior to the typical MARL algorithm in terms of reward and success rate.