Harvest-or-Transmit Policy for Cognitive Radio Networks: A Learning Theoretic Approach

We consider an underlay cognitive radio network where the secondary user (SU) harvests energy from the environment. We consider a slotted-mode of operation where each slot of SU is used for either energy harvesting or data transmission. Considering block fading with memory, we model the energy arrival and fading processes as a stationary Markov process of first-order. We propose a
harvest-or-transmit
policy for the SU along with optimal transmit powers that maximize its expected throughput under three different settings. First, we consider a learning-theoretic approach where we do not assume any
a priori
knowledge about the underlying Markov processes. In this case, we obtain an online policy using
Q-learning
. Then, we assume that the full statistical knowledge of the governing Markov process is known
a priori
. Under this assumption, we obtain an optimal online policy using infinite horizon
stochastic dynamic programming
. Finally, we obtain an optimal offline policy using the
generalized Benders decomposition
algorithm. The offline policy assumes that for a given time deadline, the energy arrivals and channel states are known in advance at all the transmitters. Finally, we compare all policies and study the effects of various system parameters on the system performance.