PEARL: Power and Delay-Aware Learning-based Routing Policy for IoT Applications

Routing between the IoT nodes has been considered an important challenge, due to its impact on different link/node metrics, including power consumption, reliability, and latency. Due to the low-power and lossy nature of IoT environments, the amount of consumed power, and the ratio of delivered packets plays an important role in the overall performance of the system. Meanwhile, in some IoT applications, e.g., remote health-care monitoring systems, other factors such as End-to-End (E2E) latency is significantly crucial. The standardized routing mechanism for IoT networks (RPL) tries to optimize these parameters via specified routing policies in its Objective Function (OF). The original version of this protocol, and many of its existing extensions are not well-suited for dynamic IoT networks. In the past few years, reinforcement learning methods have significantly involved in dynamic systems, where agents have no acknowledgment about their surrounding environment. These techniques provide a predictive model based on the interaction between an agent and its environment to reach a semi-optimized solution; For instance, the matter of packet transmission, and their delivery in unstable IoT networks. Accordingly, this paper introduces PEARL; a machine-learning based routing policy for IoT networks, which is both, delay-aware, and power-efficient. PEARL employs a novel routing policy based on the q-learning algorithm, which uses the one-hop E2E delay as its main path selection metric to determine the rewards of the algorithm, and to improve the E2E delay, and consumed power simultaneously in terms of Power-Delay-Product (PDP). According to an extensive set of experiments conducted in the Cooja simulator, in addition to improving reliability in the network in terms of Packet Delivery Ratio (PDR), PEARL has improved the amount of E2E delay, and PDP metrics in the network by up to 61% and 72%, against the state-of-the-art, respectively.