Cooperative and Dynamic UAV-Based Mobile Sink Deployment using Deep Reinforcement Learning

In maritime emergency response operations, autonomous underwater vehicles (AUVs) can perform underwater search and transmit emergency data in real-time. To collect the sensed data from AUVs across the water-air interface, the traditional method is to deploy surface relays. However, the deployment of surface relays has some disadvantages, such as poor and imbalanced communication coverage of real-time connections with AUVs. Since the deployment of static relays may limit the dynamic search area of AUVs, whereas the deployment of free surface drifting relays may disrupt the connections while the relays are drifting away with the ocean current. To address these challenges, in this paper, we propose a novel method by leveraging the flexibility of multiple unmanned aerial vehicles (UAVs) equipped with underwater hydrophones and design a dynamic deployment scheme based on a multi-agent deep deterministic policy gradient (MADDPG) approach. By this scheme, multiple UAVs are able to deploy in a cooperative manner to deal with the coverage imbalance issue and provide maximum coverage service. The numerical simulations demonstrate that the proposed scheme is effective in terms of communication coverage, fairness, and energy consumption.