Multi-Agent Reinforcement Learning-Based Delay and Power Optimization for UAV-WMN Substation Inspection

Unmanned aerial vehicles (UAV), due to their flexibility and extensive coverage, have gradually become essential for substation inspections. Wireless mesh networks (WMN) provide a scalable and resilient network environment for UAVs, where each node can serve as either an access point or a relay point, thereby enhancing the network’s fault tolerance and overall resilience. However, the UAV-WMN combined system is complex and dynamic, facing the challenge of dynamically adjusting node transmission power to minimize end-to-end (E2E) delay while ensuring channel utilization efficiency. Real-time topology changes, high-dimensional state spaces, and large solution spaces make it difficult for traditional algorithms to guarantee convergence and stability. Generic reinforcement learning (RL) methods also struggle with stable convergence. This paper introduces a new Lyapunov function-based proof to address these issues and provide a stable condition for dynamic control strategies. Then, we developed a specialized neural network power controller and combined it with the MATD3 algorithm, effectively enhancing the system’s convergence and E2E performance. Simulation experiments validate the effectiveness of this method and demonstrate its superior performance in complex scenarios compared to other algorithms.