Multidimensional Beam Optimization in Underwater Optical Wireless Communication Based on Deep Reinforcement Learning

In this work, we study learning-aided adaptive control of optical beam alignment to maintain a seamless connection with high communication performance in a point-to-point (P2P) underwater optical wireless communication (UOWC). To this end, we propose a two-step two-agent deep reinforcement learning (TSTA-DRL) algorithm that enables an underwater sensor (US) installed on the seabed to sequentially determine the beam orientation (BO) and beam divergence (BD) angles for transmitting its sensing data to an unmanned surface vehicle (USV) that may irregularly shake above the sea level. Specifically, the proposed TSTA-DRL algorithm includes two DRL agents: BO and BD. The BO agent selects the BO angle to point the optical beam of the US toward the USV to perform beam alignment between the US and USV. Moreover, given the BO angle determined by the BO agent, the BD agent chooses the BD angle to maximize the signal-to-noise ratio (SNR) while maintaining the seamless optical link between the two nodes. For the practical application of the proposed algorithm, movement data of the USV measured in the South Sea of Korea are utilized for training the proposed algorithm. The simulation results demonstrate that the proposed TSTA-DRL algorithm achieves the highest SNR while maintaining a stable UOWC link compared with the existing algorithms.