Trellis Coded Modulation for Non-Orthogonal Multiple Access Systems: Design, Challenges, and Opportunities

Capable of achieving massive connectivity and high reliability, the NOMA technique has been well recognized as a potential candidate for 5G mobile communications. Being a key technique of code-domain NOMA, codebook design has drawn great attention due to its potential to improve the coding gain of the system. Trellis coded modulation (TCM) techniques have been considered as an effective approach to optimize the coding gain through signal set expansion without causing bandwidth expansion. In this article, we propose a framework to apply the multi-dimensional TCM (MD-TCM) techniques in code-domain NOMA to jointly design the superposed codewords. Starting with a basic TCM-based NOMA system, we present the design criteria based on which joint multi-layer codeword design for code-domain NOMA is elaborated. Three TCM-based NOMA applications are extended: TCM-based NOMA for MIMO systems, TCM-based NOMA for cooperative relay systems, and TCM-based NOMA for vehicular systems. Open research issues in these scenarios are also discussed.