RL-based Small-World Internet of Drones Network for Low-Latency and Energy-Efficient Data Routing

Small-world networks are a type of complex network with reduced average path length and increased clustering coefficients when compared to regular networks. This in turn reduces the hop count for data transmission and improves the connectivity of the network. Motivated by this, in this paper, a reinforcement learning approach is proposed to create a small-world Internet of Drones network (SW-IoDN) for low-latency and energy-efficient data routing. Initially, an optimization problem in terms of transmission delay, energy consumption, and throughput is formulated. To solve the problem, the proposed reinforcement learning approach introduces the small-world characters (SWCs) into the Internet of Drones network (IoDN) to create a SW-IoDN. In this work, we consider introducing SWCs by replacing a small fraction of small-range UAV-UAV and UAV-GS links with an equal number of long-range links in IoDN. Thus, the proposed RL approach learns all possible pairs of small-range links and long-range links as state-action pairs and decides on the fraction of small-range links to be replaced with long-range links to create the SW-IoDN. We compare the performance of the proposed RL approach with the conventional SWC method, canonical particle multi-swarm (PMS) method, LEACH, and conventional shortest path routing algorithm in terms of network latency, lifetime, and packet loss rate. We also analyzed the effect of the ground station’s location and the different UAV hovering heights. The results demonstrate that the proposed RL methodology outperforms alternative techniques in terms of performance.