Reinforcement Learning Based Obstacle Avoidance for AUV Swarm in Dynamic Ocean Environment

As an essential component of 6G communication, the Internet of Underwater Things (IoUT) plays a crucial role in supporting various maritime activities. Due to the IoUT devices’ wide-area distribution and constrained transmit power, autonomous underwater vehicles (AUVs) have become extensively employed for data collection in underwater environments. Ensuring the reliability of underwater tasks, the autonomous obstacle avoidance of AUV swarms has emerged as a critical research area. The Multi-Agent Deep Deterministic Policy Gradient algorithm (MADDPG), which employs centralized training and distributed execution, is an effective method to address this problem. However, current studies often overlook the complex dynamic underwater environment, including the interference caused by ocean currents on AUV movement and the observation noise of objects in the ocean. To adapt to the interference of ocean currents on AUV movement, we establish a two-dimensional motion model for AUVs. This model enables us to transform the output of MADDPG into propulsion force and steering gear control force, effectively accounting for ocean current disturbances. To tackle the observation noise of obstacles in underwater environment, we first assume the obstacle is moving in uniform linear motion, and then use the Kalman filtering algorithm to process the observed signals. Our simulation results demonstrate the superiority of our scheme.