Reinforcement Learning and Vector-based Clustering Method to Achieve Energy Balanced in Underwater Acoustic Sensor Networks

Underwater acoustic sensor networks (UASNs) have a wide range of applications, such as water quality monitoring, ocean data collection and underwater resource exploration. In UASN applications, the management of energy is very important. Sensor nodes in UASNs are usually powered by batteries, which are very difficult to replace and replenish. Extending the life cycle by reducing the energy consumption of UASNs can reduce the network operation cost. Therefore, the energy management problem has become a popular research topic in UASNs. The efficiency of cluster routing in reducing the communication overhead between nodes in the network has been verified by dividing nodes into clusters and having the cluster header (CH) responsible for communicating with the base station and other nodes. However, if the cluster headers and paths are not properly chosen, the energy consumption of the nodes will be unbalanced, thus leading to premature death of the nodes. In this paper, a reinforcement learning and vector-based clustering (RVC) method is proposed to achieve energy balancing in UASNs. We use reinforcement learning and vector forwarding methods to select appropriate CHs and relay nodes to balance the node energy consumption based on the residual energy of the nodes, the transmission distance and the node density in the direction of the routing vector. The simulation results show that the method can effectively reduce the energy consumption of UASN nodes, achieve energy balancing, and extend the network survival cycle.