Q-Learning Algorithm Inspired Objective Function Optimization For IETF 6TiSCH Networks

Nowadays, Wireless Sensor Networks (WSNs) are effectively used in a wide range of application areas such as environmental monitoring, home automation, control of agricultural production processes, health monitoring systems, military surveillance, industrial automation systems, smart grids and smart cities. However, there is a need to develop efficient solutions to the problems encountered in WSN applications such as low latency, low energy consumption, real-time operation, high performance, reliability, minimum packet loss, and reliable routing.The IETF 6TiSCH protocol stack is defined as an extension of the IPv6 Internet protocol to support efficient, secure and scalable low power wireless sensor networks for industrial applications. The 6TiSCH protocol has a time-slotted/channel-hopping mechanism with dynamic resource requirements. However, these resources need to be reorganized in case of frequent routing path changes in the network. This leads to extra energy consumption due to additional computational overhead and higher communication cost. The routing layer of 6TiSCH networks uses the RPL. RPL protocol makes routing decisions based on objective functions.In this paper, a new solution to the problem of frequent routing path changes of nodes is presented by optimizing the RPL objective function inspired by the Q-Learning algorithm. With the developed method, real-time application problems such as high end-to-end packet transmission delay and high packet losses in the buffer, which are encountered in routing processes, are also improved. The proposed algorithm, a more stable network structure is provided by utilising the previous states and critical metrics in the WSNs and the reliability of the network are increased. Considering the PDR metric, the proposed method performs 33.6% better than the standard MRHOF and 11.2% better than the OF-SR1 algorithm when Rx is 60% in a network of 20 nodes. In a network of 40 nodes, it is 44.9% better than MRHOF and 7.7% better than OF-SR1. In addition the synchronization time of the network is improved by creating a more balanced network structure. The performance of the algorithm is tested in Cooja, Contiki-NG’s integrated simulation tool and the results are presented in tables and graphs.