Reinforcement Learning-based Directional Neighbor Discovery Algorithm in the Internet of Underwater Things

In Internet of Underwater Things (IoUT), nodes are usually deployed randomly in monitored environments. Without any infrastructure support, neighbor discovery is the basis for fast networking. This paper focuses on efficient neighbor discovery under directional transmission conditions. First, a Binary Sequence-based Neighbor Discovery Algorithm (BSND) is proposed, which uses the characteristics of binary sequences to determine the transmitting and receiving modes and the directional beam scanning sequences, and allows the nodes to complete neighbor discovery without any a priori knowledge. Secondly, a Q-learning based neighbor discovery algorithm (QBSND) is proposed, which introduces a reinforcement learning algorithm to help the nodes in decision making, and lets the nodes go into sleep at redundant time slots to save energy consumption. Simulation results demonstrate that compared with the traditional algorithm, BSND and QBSND have improved neighbor discovery rates by 7% and 10%, and reduced energy consumption by 18% and 34%.