Learning-Aided Multi-UAV Online Trajectory Coordination and Resource Allocation for Mobile WSNs

In this paper, we consider a multi-UAV enabled wireless sensor network (WSN) where multiple unmanned aerial vehicles (UAVs) gather data from multiple randomly moving sensor nodes (SNs). We aim to minimize the long-term average energy consumption of all SNs while satisfying their average data rate requirements and energy constraints of the UAVs. We solve the problem by jointly optimizing the multi-UAV's trajectories, communication scheduling and SN's association decisions. In particular, we formulate it as a multi-stage stochastic mixed integer non-linear programming (MINLP) problem and design an online algorithm that integrates Lyapunov optimization and deep reinforcement learning (DRL) methods. Specifically, we first decouple the original multi-stage stochastic MINLP problem into a series of per-slot deterministic MINLP subproblems by applying Lyapunov optimization. For each per-slot problem, we use model-free DRL to obtain the optimal integer UAV-SN associations and model-based method to optimize the UAVs' trajectories and resource allocation. Simulation results reveal that although the communication environments change stochastically and rapidly, our proposed online algorithm can produce real-time solution that achieves high system performance and satisfies all the constraints.