Exploiting Q-Learning in Extending the Network Lifetime of Wireless Sensor Networks with Holes

Geographic routing is one of the most popular routing protocols in wireless sensor networks (WSNs) due to its simplicity and efficiency. However, with the occurrence of holes, geographic routing incurs with the so-called local minimum problem that may lead to a long hole detour path as well as the traffic concentration around the hole boundary. In consequence, the network lifetime is shortened. In this paper, we aim at proposing a lightweight distributed geographic routing protocol, which can prolong the lifetime of WSNs under the hole occurrence. Our main idea is to exploiting Q-learning technique to estimate the distance from a node to the holes. The routing decision is then determined based on the residual energy of the nodes, their estimated distance to the holes, and their distance to the destination. The simulation experiments show that our protocol strongly outperforms state-of-the-art protocols in terms of the network lifetime, packet latency and energy consumption. Specifically, our proposed protocol extends the network lifetime by more than 12% compared to the existing protocols.