QMCR: A Q-learning-based multi-hop cooperative routing protocol for underwater acoustic sensor networks

Routing plays a critical role in data transmission for underwater acoustic sensor networks (UWSNs) in the internet of underwater things (IoUT). Traditional routing methods suffer from high end-toend delay, limited bandwidth, and high energy consumption. With the development of artificial intelligence and machine learning algorithms, many researchers apply these new methods to improve the quality of routing. In this paper, we propose a Q-learning-based multi-hop cooperative routing protocol (QMCR) for UWSNs. Our protocol can automatically choose nodes with the maximum Q-value as forwarders based on distance information. Moreover, we combine cooperative communications with Q-learning algorithm to reduce network energy consumption and improve communication efficiency. Experimental results show that the running time of the QMCR is less than one-tenth of that of the artificial fish-swarm algorithm (AFSA), while the routing energy consumption is kept at the same level. Due to the extremely fast speed of the algorithm, the QMCR is a promising method of routing design for UWSNs, especially for the case that it suffers from the extreme dynamic underwater acoustic channels in the real ocean environment.