Learning spectrum opportunities in non-stationary radio environments

Learning-based sensing policies for multi-band flexible spectrum use, in particular cognitive radios operating in non-stationary radio environments are proposed. The proposed policies stem from the stochastic non-stationary restless multi-armed bandit formulation of opportunistic spectrum access. The non-stationary radio environment assumed in this paper is an appropriate model for a realistic cognitive radio systems, where the obtainable data rates depend on many unknown time-varying factors. These are e.g. mobility, fading and primary user activity. The developed policies are index policies, where the index of a frequency band depends on the discounted average reward of the band and a recency-based exploration bonus. The exploration bonus encourages sensing frequency bands that have not been explored for a long time. However, there is a maximum number of time instances when any band can remain unexplored. These index policies are computationally simple making them attractive for mobile cognitive radios. In our simulation examples, we demonstrate that the proposed policies can often provide higher cumulative data rate than other existing state-of-the-art policies.