Reinforcement Learning-Based Routing Protocol for Underwater Wireless Sensor Networks: A Comparative Survey

Underwater wireless sensor networks (UWSNs) have emerged as a promising networking technology owing to their various underwater applications. Many applications require sensed data to be routed to a centralized location. However, the routing of sensor networks in underwater environments presents several challenges in terms of underwater infrastructure, including high energy consumption, narrow bandwidths, and longer propagation delays than other sensor networks. Efficient routing protocols play a vital role in this regard. Recently, reinforcement learning (RL)-based routing algorithms have been investigated by different researchers seeking to exploit the learning procedure via trial-and-error methods of RL. RL algorithms are capable of operating in underwater environments without prior knowledge of the infrastructure. This paper discusses all routing protocols proposed for RL-based UWSNs. The advantages, disadvantages, and suitable application areas are also mentioned. The protocols are compared in terms of the key ideas, RL designs, optimization criteria, and performance-evaluation techniques. Moreover, research challenges and outstanding research issues are also highlighted, to indicate future research directions.