Multi-Hop Deflection Routing Algorithm Based on Reinforcement Learning for Energy-Harvesting Nanonetworks

Nanonetworks are composed of interacting nano-nodes, whose size ranges from several hundred cubic nanometers to several cubic micrometers. The extremely constrained computational resources of nano-nodes, the fluctuations in their energy caused by energy harvesting processes, and their very limited transmission range at Terahertz (THz)-band frequencies (0.1-10 THz), make the design of routing protocols in nanonetworks very challenging. A multi-hop deflection routing algorithm based on reinforcement learning (MDR-RL) is proposed in this paper to dynamically and efficiently explore the routing paths during packet transmissions. First, new routing and deflection tables are implemented in nano-nodes, so that nano-nodes can deflect packets to other neighbors when route entries in the routing table are invalid. Second, one forward updating scheme and two feedback updating schemes based on reinforcement learning are designed to update the tables, namely, on-policy and off-policy updating schemes. Finally, extensive simulations in networks simulator-3 are conducted to analyze the performance of MDR-RL using different updating policies, as well as to compare the performance with other machine learning routing algorithms based on Neural Networks and Decision Tree. The results show that the MDR-RL can increase the packet delivery ratio and number of delivered packets, and can decrease the packet average hop count.