Optimal Frequency and Rate Selection Using Unimodal Objective Based Thompson Sampling Algorithm

Due to limited acoustic bandwidth and constrained battery life, transmission efficiency is a crucial issue in underwater acoustic communication (UAC). This paper studies the problem of joint frequency and transmission rate selection of a single link in UAC so as to maximize the link's average throughput. To handle this problem, we first describe this problem as a traditional optimization form and show the challenges located in solving it. Then we resort to the online learning theory by modeling this problem as a multi-armed bandit (MAB) framework. Through taking full advantage of the unimodality feature of the problem structure, we propose an algorithm called UOTS (unimodal objective based Thompson sampling algorithm) to solve this MAB problem. A finite-time analysis of the upper regret bound has been derived for the proposed algorithm. Several numerical results are also provided to verify the proposed algorithm and demonstrate that UOTS outperforms the current state-of-the-art algorithms. It is interesting that the performance loss of UOTS does not depend on the number of available pairs of frequency and rate, which can be much useful in the practical implementation.