Throughput Maximization for RF Powered Cognitive NOMA Networks with Backscatter Communication by Deep Reinforcement Learning

In this paper, we present a hybrid ambient backscatter communication (ABC) assisted framework for radio frequency (RF) powered cognitive radio networks (CRNs). In these CRNs, the secondary users (SUs) can actively transmit data when the primary user network (PUN) is idle, and harvest energy from the primary signal and transmit their own information over the primary signal when the PUN is busy. The proposed CRNs adopt non-orthogonal multiple access (NOMA) technique to further improve spectral efficiency. Considering two different spectrum sharing paradigms of the PUN and the SUs, we formulate two optimization problems by two Markov decision processes (MDPs) to maximize our long term throughputs for both underlay-interweave and overlay-interweave scenarios. We propose a deep reinforcement learning (DRL) based optimization algorithm, i.e., a deep deterministic policy gradient (DDPG)-based joint reflection coefficients adjustment and resource allocation (JCARA) algorithm, to solve these two non-convex problems under the two constructed MDPs without the non-causal and the statistical information about the dynamic environment a-priori. For the underlay-interweave scenario, the proposed JCARA algorithm jointly optimizes the transmit power and the reflection coefficients of the SUs, while for the overlay-interweave scenario it optimizes the above two variables plus time resource simultaneously. Simulation results clearly show the higher throughput performance of this proposed algorithm for the proposed CRNs in the comparison with other algorithms.