Performance Analysis and Fairness Maximization in NOMA Systems With Improper Gaussian Signaling Under Imperfect Successive Interference Cancellation

In this paper, we investigate a downlink non-orthogonal multiple access (NOMA) system consisting of a base station (BS) and two users, where each user decodes its own signal through successive interference cancellation (SIC). When the signal of one user, called the weak user (WU) who has a bad channel, is being decoded, the signal of the other user, called the strong user (SU) who has a good channel, acts as interference. In an interference channel, we can increase the achievable rate by applying improper Gaussian signaling (IGS). Therefore, we apply IGS to SU, aiming to mitigate the interference. We also consider imperfect SIC at SU, which is a more general assumption for a NOMA system. For the considered NOMA system model, we derive the outage probability of each user in a closed form, provide its asymptotic expression, and obtain its diversity order. We also derive the ergodic rate of each of the two users in a closed form. We propose two algorithms to maximize the fairness of the NOMA system based on the primal decomposition method. Through simulation, we verify that our analysis is correct, and confirm that applying IGS improves the fairness of the NOMA system.