Distributed Q-Learning-Based UAV-Assisted Small World Wireless Network for Energy-Efficient and Delay-Critical Data Transmission

In recent years, Machine Learning (ML) has gained much importance in many application fields. ML has been applied in multiple fields that include language processing, image classification, and communication networks. In this paper, we propose distributed Q-Iearning (DQL)-based UAV-assisted small-world network for energy-efficient and delay-critical data transmission. In the proposed work, an optimization problem is formulated in terms of energy consumption and transmission delay for a multilayer UAV-assisted wireless network. Thereafter, DQL model is considered for solving the optimization problem by introducing the long-range links into the conventional network for the introduction of small-world characteristics (SWCs). This paper considers each layer as a cluster (subnetwork) for the local training of Q-Iearning. The Q-Iearning model is trained locally on the possible long-range links among a pair of clusters and the reward is obtained in terms of SWC parameters such as average path length and average clustering coefficient. These long-range links are aggregated and trained globally on the whole network using the Q-Iearning model. Here, the reward is assigned based on energy consumption and transmission delay. Through simulations, we show that the proposed approach outperforms the other approaches such as the conventional SWC method, LEACH, and conventional shortest path method in terms of energy consumption, transmission delay, and packet delivery ratio.