Event-Driven Transformer-Based Reinforcement Learning for Trajectory Design and Channel Assignment in Multi-UAV Assisted Communication

In the assisted communication system with multiple unmanned aerial vehicle base stations (UAV BSs), it is a challenge to achieve optimized trajectory design and channel assignment in unknown environments. Multi-agent reinforcement learning has proved effective for the above policy optimization. However, most existing baseline methods suffer from the excessive communication overhead and information explosion, which results in poor performance. In this paper, a novel event-driven transformer multi-agent reinforcement learning (EDT-MARL) framework is proposed to solve the multi-UAV assisted communication problem. The event-driven listening policy optimization is designed to learn the best strategy of event-driven listening based on the current channel states, which reduces unnecessary information listening for UAV BSs effectively. The transformer mechanism is used to extract useful information and patterns from massive observational information, so that the critic network of UAV BSs can evaluate the state information more accurately. Simulation results indicate that the proposed approach attains improved convergence over the baselines. UAV BSs learn the optimal trajectory design and channel assignment strategies to achieve their maximum cumulative rewards. Additionally, considering the cooperation and competition between UAV BSs, the proposed method with monotonic convergence provides an estimated Nash equilibrium for the multi-UAV assisted communication Markov game.