A Q-Learning Approach for Real-Time NOMA Scheduling of Medical Data in UAV-Aided WBANs

Unmanned Aerial Vehicles (UAVs) have emerged as a flexible and cost-effective solution for remote monitoring of the vital signs of patients in large-scale Internet of Medical Things (IoMT) Wireless Body Area Networks (WBANs). This paper deals with the problem of using UAVs for real-time scheduling of the transmission of vital signs in delay-sensitive IoMT WBANs. The main challenge for such a network is to timely and reliably transmit the vital signs of patients to the remote monitoring center without interrupting their daily lifestyles. To achieve this goal, we propose a
$Q$
-learning-based algorithm to optimize the trajectory of each UAV, as the mobile Base Station (BS), to harvest vital signs of patients in outdoor applications, especially in unreachable areas. In this algorithm, UAVs learn to reach the best 3D position by discovering the network environment step-by-step. It stands for the position in which the covered patients by each UAV have the highest transmission rate, the least delay and energy consumption. Moreover, we employ the Non-Orthogonal Multiple Access (NOMA) technique to simultaneously schedule multiple transmissions by accepting a degree of interference between them in order to enhance the spectrum efficiency of the network. Eventually, the performance of our proposed scheme is evaluated via extensive simulations in terms of throughput, energy consumption, and delay. The simulation results show that our proposed scheme iteratively converges to the benchmark value of the mentioned factors by increasing the information of cluster environment through episodes.