Distributed Routing Optimization Algorithm for FANET Based on Multi-agent Reinforcement Learning

Recent advancements in protocol design and optimization for Flying Ad-hoc Networks have shown significant progress. However, the existing centralized optimization control algorithms pose conflicts with the distributed nature of these systems. To address this challenge, this paper proposes a novel distributed routing optimization algorithm based on multi-agent reinforcement learning, integrated with an adaptive multi-mode routing framework incorporating multi-protocol cooperation mechanisms. The algorithm adopts the Decentralized Execution Multi-Agent Deep Deterministic Policy Gradient (DE-MADDPG) algorithm framework, enabling individual unmanned aerial vehicles to directly adjust the protocols and protocol parameters of the current node based solely on local network information. This optimization enhances network structure and overall performance without considering the transmission delay of control signals. The ns3-gym simulation platform is employed for performance evaluation, comparing the proposed algorithm with Deep Q-Network (DQN), Multi-Agent Deep Deterministic Policy Gradient (MADDPG), and other algorithms. The experimental results demonstrate that compared to the routing algorithm optimized based on DQN, the proposed algorithm exhibits the best overall performance, with a 26.31% reduction in energy consumption and a 19.69% decrease in end-to-end delay.