Multi-AUV Based Underwater Target Tracking Method via Reinforcement Learning in Dynamic Ocean Environment

Autonomous underwater vehicle (AUV) has gradually been developed to showcase its significant value in performing underwater tasks. Given the current situation that single AUV is constrained with limited abilities in detection, information processing and communication efficiency, multi-AUV system is employed to complete complex tasks, among which target tracking is one application where multi-AUV system shows superior performance. However, most of the existing studies utilize simplified scenario like constant current velocity, which fails to simulate real dynamic ocean environment including current and obstacles. These factors can affect the overall path planning policies of AUVs or cause mission failure and economic loss. To improve the stability and validity of AUVs in the target tracking task, this paper introduces a multi-AUV based underwater target tracking method via reinforcement learning (RL) in dynamic ocean environment with current and obstacles. The concept of standoff circle is adopted to balance the distance between the AUVs. To be specific, we first model the ocean current and analyze the kinematic model of an AUV. Then, the target tracking task is modeled as a Markov decision process, while RL is applied to achieve navigation and position control between AUVs and the target. The numerical simulation results reveal the best success rate of the AUVs at 83%, which proves the aforementioned method can perform better tracking accuracy and achieve superior robustness.