An Extended Bandit-Based Game Scheme for Distributed Joint Resource Allocation in Underwater Acoustic Communication Networks

This paper investigates a joint discrete-channel and continuous-power allocation problem for multi-user underwater acoustic communication networks. The unknown underwater acoustic Channel State Information (CSI) and the distributed optimization requirement make the proposed hybrid discrete-continuous optimization problem full of challenges. Firstly, an adversarial multi-player bandit game model is formulated, which enables each user to independently optimize its own strategy, thereby achieving the distributed decision. In the strategic game, the Multi-armed Bandit (MAB) learning theory is exploited to achieve the best response strategy of independent user without prior CSI. Secondly, an evolutive finite discrete strategy pool learning structure is proposed to achieve an efficient search for the hybrid discrete-continuous space. The constant evolvement of strategy pool endows the proposed MAB-based algorithm with the ability to search the whole continuous power space, thereby avoiding missing the superior strategy caused by the discretization of continuous space. Thirdly, a selection probability setting rule is proposed, which promotes the exploration-exploitation balance for the dynamic strategy pool, thereby improving the learning efficiency. Finally, simulation results demonstrate the superiority of the proposed algorithm.