A Distributed and Adaptive Routing Protocol for UAV-aided Emergency Networks

Due to its strong flexibility, easy deployment, high maneuverability and extensive connectivity, unmanned aerial vehicle (UAV) swarm has been widely used in the construction of emergency communication network in recent years. Among them, packet routing in a resilient and adaptive manner is one of the fundamental problems for cooperation between multiple UAVs to complete search and rescue tasks. Recently, reinforcement learning (RL) technique has provided a new opportunity for network-related applications, including routing. However, most existing RL-based routing protocols suffer from issues such as local optimum, blind exploration and slow convergence speed. Additionally, the routing protocols based on deep reinforcement learning (DRL) has high computational complexity, making them unsuitable for energy-limited emergency relief scenarios. In this paper, we proposed a Q-learning aided resilient routing protocol with hindsight pre-calculation (QR
²
HPC) in UAV swarm for the construction of the emergency networks. Firstly, a dynamic exploration and exploitation coefficient is proposed based on the number and speed of neighbors. Secondly, a warm-start mechanism is proposed in the exploration phase that modifies the traditional random next hop selection to a routing approach guided by various indicators. Finally, we introduce a hindsight pre-calculation (HPC) mechanism to improve the robustness of Q-table to traffic flow changes. The experimental results manifest that our protocols can make effective routing decisions in dynamic wireless multi-hop networks, thereby enhancing the system performances in terms of packet delivery ratio, end-to-end delay, throughput and network lifetime.