Multi-armed bandit based opportunistic channel access: A consideration of switch cost

In this paper, we study on the problem of opportunistic channel access without prior information about channels. We model it as the multi-armed bandit (MAB) problem. There are N independent arms. The player can choose one arm to play each time and get a reward. Switch cost is taken into consideration when player switches arm. Switch cost includes reward loss and switch delay. The concept of regret is used to measure the performance of an access policy. We prove that the regret of Lai-Robbins policy with switch cost grows with time at logarithmic order as that without switch cost, though with a much higher leading constant. Then we propose a policy referred as reducing switch with advanced play (RSAP), whose regret is shown to grow with time at logarithmic order with a much smaller leading constant.