DRL-Assisted Dynamic Subconnected Hybrid Precoding for Multi-Layer THz mMIMO-NOMA System

Massive multiple-input multiple-output (mMIMO) techniques can be combined with the non-orthogonal multiple access (NOMA) scheme in terahertz (THz) communication to achieve multiplexing gains and satisfy the ultra-high capacity and massive connectivity requirements. However, the development of a near-optimal solution for energy and spectral efficiency problems in a dynamic wireless cellular environment remains challenging. In this paper, a cooperative THz mMIMO-NOMA enabled base station is established to optimize the power consumption and maximize the spectral efficiency. A multi-layer mMIMO antenna architecture is used to perform dynamic sub-connected hybrid precoding in each layer. The fuzzy c-means clustering algorithm is used to group densely located users into clusters to efficiently use the power coefficients. To optimize the power distribution constraints and coordination of the hybrid precoding structure, a multi-agent deep reinforcement learning algorithm is developed, which operates in a distributive manner. Each base station layer involves an agent that trains a deep Q-network, and optimal actions are executed by sharing exchangeable network parameters among layers. The simulation results indicate that the proposed scheme is able to learn the trade-off between maximization of the energy efficiency and overall system capacity.