Threshold Based Signal Detection and the Average Symbol Error Probability for Downlink NOMA Systems With M-ary QAM

Non-orthogonal multiple access (NOMA) is a technique which allows multiple users to share the same radio access resources. However, there exist interferences among these users in the NOMA system. These interferences need to be taken into account in the signal detection and error performance analysis. In the downlink NOMA system, the far user detects its signal by treating the near user’s signal as interference, while the near user employs successive interference cancellation (SIC) to detect its signal. Based on the conventional signal detection, it is not straightforward to derive the error performance for both the far user and near user. By analyzing the constellation relationships between the far user’s signal and superposition coded (SC) signal, and the near user’s signal and SC signal, in this article, a simple threshold based detection algorithm is proposed to detect both the far user’s signal and near user’s signal. Furthermore, by aid of the constellation relationships, in this article, simple expressions of the average symbol error probability (ASEP) with squared $M$ -ary quadrature amplitude modulation for both the far user and near user are easily derived in Rayleigh fading channels. Simulation results validate the derived theoretical ASEP results.


I. INTRODUCTION
Nowadays there is an ever-growing demand for high data transmission rates and reliability in wireless communication systems. Traditionally, multiplexing is a technique to improve data transmission rate. Multiple-input multiple-output (MIMO) is a technique to provide multiplexing gain in wireless communications. Recently non-orthogonal multiple access (NOMA) has been considered as a promising technique to improve data transmission rate in 5G wireless communication system [1], [2]. In the NOMA system, all users share the same radio access resources, thus allowing a further improvement in spectral efficiency. In the conventional NOMA system, superposition coding is employed at the transmitter in order to enable sharing of the common resources for different users, and successive interference cancellation (SIC) is used to detect the superposition coded (SC) signals at the receiver [2], [3].
There are two kinds of NOMA systems, uplink NOMA (UL-NOMA) and downlink NOMA (DL-NOMA) systems.
The associate editor coordinating the review of this manuscript and approving it for publication was Md. Arafatur Rahman .
The research problems in NOMA systems can be mainly categorized into error performance analysis [4]- [8], outage probability analysis [9], [10], power allocation [11] and system capacity or ergodic sum-rate [12], [13]. In this article, we mainly focus on error performance analysis of the DL-NOMA systems. In general, the DL-NOMA system contains one base station (BS) and two users, a far user and a near user. Due to the non-orthogonal multiple user access there is an interference at the receiver for either the far user or the near user. Most of the work in the literature has taken into account the interference to detect the users' signal and to analyze the error performance. In the literature, the conventional signal detection is that the far user detects its signal by regarding the near user's signal as interference, and the near user uses SIC to detect its signal. In [4] and [7], the error performance of the DL-NOMA system is analyzed based on the conventional signal detection. The derivation of the error performance is also based on two dimensional signal constellations and the procedure to derive the error performance is not concise. Since an M -ary quadrature amplitude modulation (M QAM) symbol is composed of independent in-phase and quadrature components, [5] firstly analyzed error performance based VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ on only the one dimensional in-phase or pulse amplitude modulation (PAM) signal. The derivation of error probability for the far user is based on the relationship between the SC signal and the far user's signal and is not straightforward. However, the derivation of error performance for the near user is too long because by examining and comparing the decision regions and distant parameters, [5] classifies all signals into three types. Then, by considering the relationship between PAM and QAM, [5] derived error performance expressions for M QAM. However, these expressions are only derived for additive white Gaussian noise (AWGN) channels and not fading channels. Meanwhile, the work of [6] also analyzed the error performance based on interference cancellation. Since the probability density function (PDF) of the summation of interference and noise is not Gaussian distributed, the error performance expression is not simple. Finally, taking into account interference, [8] derived the error performance based on one dimensional signals. The average symbol error probabilities (ASEPs) for both the far user and near user are derived in Rayleigh fading channels considering imperfect SIC. However, the derived error performance expressions are too complicated.
In summary, most of the above work considered interferences to derive error performance but the derivations are not concise. In this article, we will not take into account the interference to derive the error performance. Instead, by analyzing the constellations of the individual user's signal and the SC signal, we firstly propose a threshold based simple signal detection algorithm for both the far user and near user. Then aided by the proposed detection algorithm, we easily derive the ASEPs for both the far user and near user.
Hence, the main contributions of this article are summarized as: 1) A threshold based simple signal detection for both the far user and near user is proposed. The proposed threshold based detection has low detection complexity compared to the conventional detection. 2) An approach to derive error performance for the individual user is proposed. The approach is to convert the derivation of error performance for the individual user's symbols into the error performance of the SC symbols. The remainder of the paper is organized as follows: In Section II, the system model is presented. The constellation analysis of the individual users' symbol and the SC symbols is presented in Section III. The conventional detection scheme and the proposed threshold based detection algorithm are presented in Section IV. The ASEPs for both far user and near user are derived in Section V. In Section VI, the numerical results are demonstrated. Finally, the paper is concluded in Section VII.
Notation: Bold lowercase and uppercase letters are used for vectors and matrices, respectively. [·] T , (·) H , | · | and · F represent the transpose, Hermitian, Euclidean and Frobenius norm operations, respectively. E{·} is the expectation operation. j = √ −1 is a complex number. A complex symbol x has the real or in-phase part x I and imaginary or quadrature part x Q , such that x = x I + jx Q .

II. SYSTEM MODEL
The conventional DL-NOMA system has been documented in the literature [4]. The conventional DL-NOMA system consists of one BS and two users which are denoted as User n and User f . User n is an intra-cell user or near user, while User f is a cell-edge user or far user. Relatively, User n experiences good channel conditions, while User f experiences severe channel conditions. In the DL-NOMA system, the BS communicates with User n and User f on the same radio access resources, frequency and time through superposition coding. Let u be the signal set of the squared QAM with modulation order M u , u ∈ [n, f ], x n and x f be the modulated symbols to be transmitted to Users n and f , x n ∈ n and x f ∈ f . Similar to the discussion in [4], in this article, we will consider M f ≤ M n because User f experiences severe channel conditions. In the system model, it is also assumed that E{|x n | 2 } = E{|x f | 2 } = ε. By multiplying x n and x f with different power coefficients and summing them, the SC signal x sc is given by: where In the conventional DL-NOMA system, it is also assumed that both User n and User f have a single receive antenna [4]. However, in this article, we extend the single receive antenna into N r receive antennas at the receivers of User n and User f . The received signal at User u is given by: where y u ∈ C N r ×1 is the signal vector received at the receiver of User u. h u ∈ C N r ×1 is the channel coefficient vector between the BS and User u. w u ∈ C N r ×1 is the AWGN vector. The entries of both h u and w u are mutually independent and identically distributed (i.i.d.) complex Gaussian random variables (RVs) distributed as CN(0, σ 2 u ) and CN(0, ε ρ ), where ρ is the signal-to-noise ratio (SNR) at each receive antenna.
It is also assumed that σ 2 n > σ 2 f because User n experiences good channel conditions, while User f experiences severe channel conditions. It is further emphasized that the key idea in the DL-NOMA system is to allocate more power to the user with severe channel conditions [5]. Thus α f > α n is held in (1).

III. CONSTELLATION ANALYSIS OF THE SC SYMBOLS
The constellation of the SC symbols is essentially important to signal detection and error performance analysis. In the DL-NOMA system, the individual User u only receives h u x sc + w u , not their own signal h u x u + w u . The User u needs to detect or extract x u from h u x sc + w u . That motivates us to analyze the signal relationship between x sc and x u . In this section, we focus on constellation analysis of the SC symbols. Since QAM is composed of independent in-phase and quadrature components, the constellation of the SC symbols is also composed of independent in-phase and quadrature component constellations. In this section, we only analyze a one dimensional constellation of the SC symbol, either in-phase or quadrature component. Let (1) can be rewritten as: Based on superposition coding of (4), the constellations of x p f , x p n and x p sc are shown in Figure 1. The constellation pattern of the SC symbol depends on power allocation in the DL-NOMA system. This article will not discuss the power allocation in superposition coding. However, in order to make the DL-NOMA system work well, the power allocation in superposition coding of (4) must guarantee Z sc 1+lK n > Z sc lK n . This is very important to signal detection.
As discussed in Section II, the guideline of the power allocation is α f > α n and α f + α n = 1. Figure 1 shows the constellation of x p sc with α f = 0.75 and α n = 0.25, while Figure 2 shows the constellation of x p sc with α f = 0.6 and α n = 0.4. Both power allocation schemes meet α f > α n and α f + α n = 1. In this example, from Figure 1 it is seen that Z sc 5 > Z sc 4 . Figure 1 also shows that Z sc However, the constellation pattern in Figure 2 shows that Z sc 5 < Z sc 4 , not Z sc 5 > Z sc 4 which will not make the DL-NOMA system work well.
Now we discuss the constellation relationship between x p sc In this article, the above constellation relationships actually represent the equivalent models for the error performance analysis of the DL-NOMA system.

IV. SIGNAL DETECTIONS
In the DL-NOMA system, it is assumed that the channel state information (CSI) h u is known at the receiver of User u. Based on the CSI, User n and User f detect their own symbol in different ways. In the following subsections, we discuss the signal detections for User n and User f , respectively. We firstly discuss the conventional detection for the far and near users, and then discuss the proposed threshold based detection for the far and near users.

A. CONVENTIONAL FAR USER SIGNAL DETECTION
The conventional far user signal detection has been described in [4] and [5]. The far user, User f , directly detects x f by regarding the near user signal, x n , as interference.
By regarding x n as interference, (2) may be rewritten as: wherew f = h f √ α n x n + w f . Since h f is known at the receiver of User f , we further have: where According to the maximum likelihood (ML) detection rule, the estimation of x f at the receiver of User f is given by [5] (8)

B. SIC BASED NEAR USER SIGNAL DETECTION
At the receiver of User n, User n also needs to estimate the transmitted x f . Similar to the above discussion in the previous subsection, the estimation of x f at the receiver of User n can be obtained. Let the estimation of x f at the receiver of User n bex f . Then User n uses SIC to subtractx f from y n . Let y n = y n − h n √ α fxf .ŷ n can be written as: Again, since h n is known at the receiver of User n, we further have: where Based on the ML detection, the estimation of x n is finally given by:x n = min From the above signal detection for both the far user and near user, the following is evident: 1) Equs. (8) and (11) represent the ML detection. The detection complexity of (8) and (11) can be reduced by detecting the in-phase and quadrature components of x f and x n , respectively. 2) In (6),w f = h f √ α n x n + w f . The distribution of x n is uniform, while the distribution of each entry in w f is Gaussian distributed. It is not easy to derive the PDF for the entry ofw f , and to further analyze the error performance of x f . Motivated by the above observation and the constellation relationship between x p sc and x f u , in the following subsections, we present a simple and low complexity detection algorithm, threshold based signal detection, to detect the in-phase and quadrature components of x f and x n .

C. THRESHOLD BASED FAR USER's SIGNAL DETECTION
Since both the received signal y f and h f in (2), are known at the receiver of User f , we have: where . Following (4), we rewrite (12) as: As discussed in Section III, there are K n possible symbols of For convenience in discussion in the next subsection, we set t f 0 = −∞ and t f K f = ∞ in (14). For the above example, only one threshold, t f 1 is also shown in Figure 3. Generally, based on z p f in (13), and the threshold t f k in (14), the estimation of x p f can be easily implemented based on Algorithm 1:

D. THRESHOLD BASED NEAR USER's SIGNAL DETECTION
Similar to the discussion in the previous subsection, the received signal at the receiver of User n is given by: These thresholds t n k , k ∈ [1 : K f × K n − 1] are shown in Figure 3. Based on the proposed threshold based detection algorithm, the estimation of x p n takes two steps.
Step 1:Apply the proposed threshold based detection algorithm in the previous subsection to find t , which is shown in Algorithm 2.
Step 2:  Compared to the detection in (8), the proposed Algorithm 1 will not treat x n as an interference to detect the far user's signal. The proposed Algorithm 1 only requires comparison, and not the norm operation. Hence, the detection complexity of Algorithm 1 is less than (8).
Compared to the SIC based detection of x n in (11), the proposed Algorithm 2 and 3 do not use SIC and/or norm operations to detect the near user's signal. The proposed Algorithm 2 and 3 also only need comparisons. Hence, once again the detection complexity of Algorithms 2 and 3 is much lower than (11).
In the following discussion, we regard that the complexity of the conventional detection and the proposed threshold based detection is in terms of real operations. It is also assumed that one complex operation is equal to four real operations averagely [14].
In (7), h H f y f takes N r complex multiplications and N r − 1 complex additions, which is equal to 4(2N r − 1) real operations. It is easy to calculate that h f 2 F √ α f in (7) takes 4N r real operations. Finally, the division (7), needs two real operations. Totally from y f to (7), it takes 12N r − 2 real operations.
For the far user's signal detection, (7) also can be rewritten as: where p ∈ [I , Q]. Based on the ML detection, the estimation of x p f is given by: In (18), given a x p f ∈ p f |z p f − x p f | 2 costs two real operations. If we use the sort(·) function to implement the function min(·) in (18), the complexity is K f log 2 K f real operations [15]. (18) needs 2K f + K f log 2 K f real operations for inphase or quadrature signal detection. For the conventional detector to detect one complex far user signal, the overall number of real operations is given by: VOLUME 8, 2020 However, for the proposed threshold based detector to detect one complex far user signal, the overall number of real operations is given by: In the conventional detection, User n detects the far user's signal first, and then detects its signal based on SIC. From the above discussion, the near user needs δ f c real operations to detect the far user's signal. In (9), y n − h n √ α fxf costs N r complex multiplications and N r subtractions, which is equal to 8N r real operations. Similar to the above discussion, from y n to (10), it takes 12N r −2 real operations. For the near user's signal detection, (11) also can be rewritten as: Based on the ML detection, the estimation of x p n is given by: (22) needs 2K n + K n log 2 K n real operations for in-phase or quadrature signal detection. For SIC based detector to detect one complex near user signal, the overall number of real operations is given by: For the proposed threshold based detection, Algorithm 2 needs K f − 1 real operations, and Algorithm 3 needs K n − 1 real operations. For the proposed threshold based detector to detect one complex near user signal, the overall number of real operations is given by: Define the percentage of complexity reduction for the proposed threshold based detection compared to the conventional detection as: where u ∈ [f , n]. β f and β n are the percentage of complexity reduction for the far user and near user, respectively. Again, as an example, we set K f = 2 and K n = 4. The proposed threshold based detection results β f = 45.45% and β n = 75% complexity reduction for N r = 1, and β f = 29.41% and β n = 71.15% complexity reduction for N r = 2, respectively.
From the above complexity analysis and discussion, it is easy to find that the proposed threshold based detection has low detection complexity compared to the SIC based conventional detection.
Finally, the error probability p n (e) over the AWGN channel is given by:

C. ASEPs OF FAR AND NEAR USERS IN RAYLEIGH FADING CHANNELS
The SEP of both User f and User n over AWGN has been shown in (28). In this subsection, we will derive the symbol error probability of both User f and User n in Rayleigh fading channels based on the received signal in (2). Given h u , p s u (e) in (28) becomes p s u (e|h u ). ρ in (34) and (38) becomes h f 2 F ρ and h n 2 F ρ, respectively. Define the instantaneous SNR at the receiver of User u as γ u = P s P n = h u 2 F ρ, where P s is the instantaneous signal power and P n is the instantaneous noise power. The PDF of γ u is given by [16]: whereγ u = σ 2 u ρ. Finally, the ASEP of User u in the Rayleigh fading channel is given by: where p s u (e|h u ) = 2p u (e|h u ) − p 2 u (e|h u ). In order to derive closed-form ASEP expressions, we need to approximate the Gaussian Q-function in (40) for integration. Taking into account conciseness of ASEP expressions and approximation accuracy we approximate the Q-function in 2p u (e|h u ) using the trapezoidal rule, while we approximate the Q-function in p 2 u (e|h u ) using the exponential bound in [17]. The Gaussian Q-function based on the trapezoidal rule may be approximated as: where s k = sin 2 (kπ/(2c)) and c is the number of partitions for the integration in the Q-function.

VI. NUMERICAL RESULTS
In this section, we present the simulated symbol error rate (SER) versus SNR results for the DL-NOMA system. In the simulation, it is assumed that the channel fading coefficients h f and h n with AWGN are the same as discussed in Section II. Both N r = 1 and N r = 2 are considered in this article. The remaining simulation parameters are tabulated in Table 1.
It is further assumed that the CSI is fully known at the receiver of User f and User n. We also calculate the theoretical results of (43) and validate these theoretical results by simulations.   ''1 × N r M f -M n QAM User u Csimula, simula or theory'', 1 × N r denotes that the BS is equipped with one transmit antenna, and User u is equipped with N r receiver antennas; M f -M n QAM denotes that the symbols to be transmitted to User f and User n are M f QAM and M n QAM, respectively; ''Csimula, simula or theory'' denote simulation based on conventional detection, simulation based on proposed threshold based detection and theoretical results based on (43), respectively.
From Figures 4 and 5, it is observed that the symbol error performance of the proposed threshold based detection is identical to the one of the conventional detection. It is also observed that the theoretical results are worse than simulation results at very low SNR. This is caused by the low accuracy of the approximated Q-function using the exponential bound.
However, it is also observed that the theoretical results match very well with the simulation results for other SNR ranges for two DL-NOMA systems with N r = 1 and N r = 2.

VII. CONCLUSION
In this article, we investigated the signal constellation relationship between the individual user's signal and the SC signal. Based on the constellation relationships, we proposed a simple signal detection scheme, threshold based detection for both far and near users. Compared to the conventional detection, the proposed threshold detection does not treat the near user's signal as interference at the receiver of the far user, and also does not use SIC to detect its signal at the receiver of the near user. The detection complexity for both the conventional detection and the proposed threshold based detection was analyzed. We further derived closed-form ASEP expressions for both the far user and near user based on the constellation relationships. Finally, we presented the simulation results which served to validate the theoretical frameworks. In future, we will directly extend the proposed threshold based detection algorithm and the derivation of the ASEP into multi-user DL-NOMA system with multiple receive antennas. We may also extend the work of this article into other channel fading models.