Energy Efficient Beamforming Design for Non-Orthogonal Multiple Access Systems: A Curiosity-Driven Approach

In this work, beamforming design and resource allocation for overload non-orthogonal multiple access (NOMA) systems is investigated. Based on this, a pure-NOMA framework is proposed to benefit from the non-orthogonal resources. In addition, an energy efficiency (EE) maximizing problem is formulated by jointly designing the beamforming, user grouping, as well as power allocation of users. Reinforcement learning (RL) has been shown to be very effective in tackling this kind of joint optimization in wireless communication networks. However, the high dimensionality and coupling non-convex mixed integer nonlinear programming (MINLP) problem makes conventional RL methods hard to get an ideal reward in practice. Given this challenge, a curiosity-driven solution is proposed to meet the MINLP challenge. Numerical results indicate that: 1) the pure-NOMA framework provides extra room for DoF and capacity improvement, which results in 31.58% and 25% reward gain of DRL-based method and curiosity-driven method; 2) The curiosity-driven approach enabled 14.78% and 13.33% reward gain compared with the DRL-based method in hybrid-NOMA and pure-NOMA schemes separately; 3) Simulations in scenarios with various quality of service (QoS) requirements demonstrated the maximum gain at 82.98% time with extra time cost less than 5%.