Relay Selection for Underwater Acoustic Sensor Networks: A Multi-User Multi-Armed Bandit Formulation

Multi-user cooperative transmission is an attractive architecture for underwater acoustic sensor networks (UASNs). Cooperative transmission depends on careful allocations of resources such as relay selection, but traditional relay selection requires precise measurements of channel state information, which is infeasible for multi-user cooperative transmission due to the unique features and hardware restrictions of UASNs. In this paper, we model multi-user relay selection under a multiuser multi-armed bandit (MU-MAB) framework, whereby users are not provided any prior knowledge about underwater acoustic channel conditions. We first exploit a novel MU-MAB algorithm, DSMU-MAB, for relay selection, assuming that the reward distributions are initially unknown but remain constant. Second, we consider an evolving environment in which the reward distributions undergo changes in time, and DSMU-rMAB, a derivative of DSMU-MAB, is proposed, which can be robust to abrupt changes in underwater communication environments. The proposed algorithms not only help sources find the suitable relays to achieve a high quality transmission and avoid collisions among users but also reduce the mass of information exchanged among users. We established the effectiveness of our proposed algorithms using theoretical and numerical analyses.