Contextual Multi-Armed Bandit-Based Dynamic Cooperative Link Configuration for AUV in UWASNs With Energy Harvesting

This paper investigates the cooperative link configuration problem for Autonomous Underwater Vehicle (AUV) in Underwater Acoustic (UWA) sensor networks with Energy Harvesting (EH), which aims to maximize long-term cumulative capacity by jointly optimizing cooperation relay, AUV transmission power, and relay transmission power. Subject to unknown time-varying Channel State Information (CSI), unpredictable stochastic EH, and variable communication topology caused by AUV mobility, the proposed problem is an unknown dynamic combination decision optimization problem. To resist the change of the dynamic UWA communication topology with unknown time-varying CSI and EH, a novel collaborative contextual multi-armed bandit learning framework is proposed, which allows relay nodes to learn cooperatively and enriches the learning information of relays. Consequently, the proposed learning framework can resist the change of the dynamic UWA communication topology efficiently, thereby achieving the superior link configuration quickly. Besides, a learning-rate adjustment rule for the dynamic UWA communication topology is proposed to adaptively balance the exploration-exploitation, thereby avoiding missing the real superior joint relay-power configuration. Finally, simulation results show the significantly superiority of the proposed scheme.