OptRISQL: Toward Performance Improvement of Time-Varying IoT Networks Using Q-Learning

In order to support the recent explosive growth in the applications of Internet of Things (IoT), networking technologies are evolving, resulting in high data throughput, low-latency data transfer, and improved lifetime of Internet of Things Devices (IoDs). These technologies work fairly for static conditions as the devices have fixed locations. However, in several practical IoT networks, including intelligent transportation networks and mobile health monitoring systems, the devices change their locations with time, resulting in time-varying network topologies. Dynamic networks generally operate on multi-hop data transmission schemes. However, due to its dynamic nature, these networks are susceptible to poor performance as a consequence of the inaccurate selection of relay IoD. In this context, the selection of optimal relay IoD towards data transfer is an important problem in time-varying IoT networks. To address such a critical issue, in this work, we consider a dynamic IoT network in which the devices select an optimal relay IoD at various discrete time instants to improve network performance. Thereafter, a novel reinforcement learning-based data routing algorithm in the time-varying multi-hop IoT network is proposed for optimum data routing. The proposed algorithm, Opt imal
${R}$
elay
${I}$
oD
${S}$
election Using Q-L earning (OptRISQL), selects the optimum relay IoD for data routing using Q-learning. The proposed method maximizes the aggregate reward value between specified device-gateway pairs by adjusting the network’s Q-matrix at discrete time instants to identify optimal relay IoD. The proposed method’s applicability and effectiveness are demonstrated using a simulated IoT testbed and real-field datasets. Moreover, when compared to various existing methods, the acquired findings indicate the proposed method’s improved network performance in terms of Energy-Efficiency (EE) and Quality-of-Service (QoS).