Q-learning based spectral hole detection with adaptive sampling point and sensing threshold for broadcasting services

Recently, the spectrum resources of broadcasting services are increasingly scarce due to the urgent requirement of high definition (HD) and ultra-high definition (UHD) broadcasting, leading to a continuous increase in the demand for spectrum. Spectral hole detection seems extremely important due to its capacity of ide spectrum detection for possible spectrum reuse. However, the performance of classical spectral hole detection schemes is limited by the difficulty of threshold setting and sampling point selection. To improve the spectrum utilization further, this paper proposes a Q-learning based spectral hole detection scheme, where the sampling of received signal and sensing threshold adaptively varies with the radio environment. The optimal sampling point and sensing threshold are determined by Q-learning to minimize the predefined utility function. The effectiveness of the proposed scheme has been confirmed.