A Time-Slotted Data Gathering Medium Access Control Protocol Using Q-Learning for Underwater Acoustic Sensor Networks

Contention-based medium access control (MAC) protocols for underwater acoustic sensor networks are designed to handle packet collisions that are caused by long propagation delays. However, existing protocols are known to suffer from relatively high collisions, which decrease system performance. To enhance system performance, we propose a contention-based MAC protocol that employs a widely-popular machine learning technique, namely, Q-learning. Using Q-learning, the proposed protocol allows the sensor nodes to intelligently select the back-off slots and accordingly schedule the transmission of data packets such that collisions are minimized at the receiver. Unlike in existing protocols, the sensor nodes are not required to exchange scheduling information, which implies that the proposed protocol has low complexity and overhead. Under varying traffic loads and node numbers, the proposed protocol is compared with the state-of-the-art ALOHA-Q for underwater environment (UW-ALOHA-Q), multiple access collision avoidance for underwater (MACA-U) and exponential increase exponential decrease (EIED) protocols. Results demonstrate the effectiveness of the proposed protocol in terms of energy efficiency, channel utilization, and latency.