A Reinforcement Learning Approach for Energy Efficient Beamforming in NOMA Systems

As an advanced version of the Internet of Things (IoT), the Internet of Everything (IoE) network enables massive access for users and machine-type devices. A surge in the number of connected devices results in a stringent requirement for multiple access. To meet this challenge, a beamforming scheme in a downlink multiple-input and single-output (MISO) transmission is investigated, where an overloaded base station (BS) is committed to serving every user. Due to the deficiency of the number of antennas, the overload user equipment (OUE) needs to ride on an existing beam to connect to the BS. The optimization goal in this work is to maximize the energy efficiency (EE) of the system by joint optimizing for the selection of the shared beam and power allocation, subject to a constraint of the total power allocated to each beam. To tackle the formulated problem, a proximal policy optimization (PPO) based solution is proposed for jointly determining the beam choice and power allocation. Simulation results revealed that an average EE gain of 27.75 % can be achieved by the proposed solution compared with a spectral efficiency (SE) based method. Furthermore, the simulation with an increasing number of overload users highlights the performance of the joint optimization.