Deep Learning-based Joint Optimization of Modulation and Power for Nonlinearity-Constrained System

For wireless communication systems with a long distance or severe interference, the insufficient transmit power limits the system performance. In this case, the maximum transmit power depends on the nonlinearity and the saturation region of the power amplifier (PA), which is referred to as a nonlinearity-constrained problem in this paper. To increase the transmit power as high as possible in a nonlinearity-constrained system, this paper proposes an autoencoder-based system to jointly optimize the modulation scheme and transmit power. The optimal solution can achieve a tradeoff between increasing the transmit power and reducing the nonlinear distortion. Meanwhile, the optimized signal constellation and the neural network-based receiver can effectively improve the capacity against nonlinear distortion. The simulation results indicate that the proposed method outperforms conventional methods in terms of symbol error rate (SER) and transmit power, and the SER of the proposed method is close to the SER lower bound of the nonlinear PA.

DPD technique, a digital-domain predistorter is added before the PA to compensate for the nonlinearity of the PA. In [14], a post-distortion algorithm is proposed for orthogonal frequency division multiplexing (OFDM) systems by employing the nonlinear components of the received signal to improve detection performance. Another symbol detection improving method is proposed in [15], in this paper, the author proposes a DNN-based symbol detector for highly dynamic channels (HDCs) and a basis expansion model to reduce the network size dramatically while achieving similar performance. Both of the proposed model have better performance for HDCs compared with the traditional MMSE method. Some other post-distortion algorithms are realized by using a learning method. For example, in [16], the effect of nonlinear distortion on Quadrature Amplitude Modulation (QAM) signals is investigated, and a bit-level demodulator network (BLDnet) using deep neural network (DNN) is proposed for nonlinear compensation. BLDnet consists of a fully connected neural network, and it is performed at the receiver to reduce the binary cross-entropy function, thus improving the bit error rate (BER) or symbol error rate (SER) performance. Furthermore, nonlinear interference models such as fiber channel models can be embedded with an autoencoderbased DNN. Through optimizing the mean square error (MSE) between the input and output of the autoencoder, the geometric constellation shapes can be learned under an average power constraint [17]. In [18], the method of overcoming the nonlinearity of PA by optimizing the amplitude and phase of desired beams is proposed, in which the PA nonlinearity is first considered in beamformers. Therefore, the method is useful in redesigning the beamformers for nonlinearityconstrained systems.
Some other methods against nonlinearity focus on reducing the peak-to-average power ratio (PAPR) through coding and modulation schemes. In [19], an autoencoder-based block error rate (BLER) reduction network is proposed for OFDM systems. Through gradient-based training, the network can achieve lower BLER with a coding method. In [20], an autoencoder-based DNN encoder is proposed to reduce PAPR in OFDM systems. However, since the loss function consists of the MSE between the input and output of the autoencoder, and the PAPR of the transmitted signals, this method can achieve lower PAPR and BER.
On the other hand, the transmit power control methods, such as power back off (PBO) enable the PA to work in the linear region. These methods are widely used in narrowband communication systems, such as the Global System for Mobile (GSM) communication system. However, with the increase of PAPR, the power efficiency of the PBO method dramatically decreases, thus leading to a lower power efficiency of the PA.
It should be noted that the above-mentioned methods have their own limitations in the optimization for nonlinearityconstrained system. The PBO method only optimizes the power control without optimizing signal design, while the other methods optimize the signal design without optimizing the power control.
In this paper, the method of optimizing the modulation and power control for the nonlinearity-constrained system simultaneously is proposed. This paper addresses the nonlinearityconstraint system design problem by jointly optimizing the modulator, the transmit power, and the demodulator. Theoretically, signal waveform design can be considered as a vector set design problem, and its complexity increases exponentially as the vector length increases. To facilitate the modulator design, this paper focuses on the constellation design problem in a single carrier frequency division multiple access (SC-FDMA) system with a low PAPR. As shown in Fig. 1, the signal waveform of SC-FDMA can be regarded as an orthogonal time-division multiplexing signal. Therefore, its robustness against nonlinearity is mainly affected by the constellation shaping of the symbols, so the waveform optimization problem can be simplified to a constellation shape optimization problem. To realize joint optimization of constellation shaping, transmit power, and demodulator, an Autoencoder is proposed, which consists of a constellation mapping module, a power control unit, and a neural networkbased receiver.
The rest of this paper is organized as follows. Section II describes the system model. Section III describes the algorithm of the SC-FDMA Autoencoder. Its performance is evaluated in Section IV, and Section Vconcludes this paper.

II. SYSTEM MODEL
The structure of the proposed nonlinear-constrained communication system based on the SC-FDMA Autoencoder is shown in Fig. 2.
First, the information source is transformed into complex modulated symbols through the designed signal constellation. Then, using a serial to parallel converter, the modulated symbols are grouped into blocks with a length of symbols, i.e., Afterward, each block is passed through an N 0 -point discrete Fourier transform (DFT) to produce the frequencydomain signal vector and its elements can be expressed as (3) In localized frequency domain multiple access (LFDMA), S 1 is fed into a set of consecutive subcarriers from N 0 subcarriers (N 1 > N 0 ) to obtain and its elements can be expressed as where f 0 is the starting index. The frequency-domain signal S 1 are then passed through an N 1 -point inverse discrete Fourier transform (IDFT) to produce s 1 before parallel to serial conversion, which can be expressed as The elements of s 1 can be expressed as The discrete symbols are then processed by a shaping filter and converted into the analog domain for radio frequency (RF) up-conversion. The up-converted signal s 3 (t) is then amplified by variable gain amplifier (VGA) to obtain where P is the power of the digitaldomain signal; c(P ) is the analog or digital control signal, which is adjusted by the output of the power control unit of SC-FDMA Autoencoder. s 4 is then fed into the PA to obtain the transmit signal x(t).
The channel is modeled as a multi-path fading channel. The received signal r 1 (t) is filtered and converted into a digital domain after down-conversion. After the DFT demodulation and subcarrier demapping, the signal R 2 is equalized by a zero-forcing (ZF) or minimum mean square error (MMSE) frequency-domain equalizer (FDE). After passing through the IDFT demodulator, r 2 is fed into a neural network-based demapping module, and r 3 is finally obtained in the form of a M -dimensional vector, where M is the modulation order.

III. THE ALGORITHM OF SC-FDMA AUTOENCODER
The proposed SC-FDMA Autoencoder consists of three modules: a constellation mapping module, a power control unit, and a neural network-based demapping module. In the training process of the SC-FDMA Autoencoder, the training data is encoded as a one-hot vector with M elements. Then, the one-hot vector is fed into the power control unit and the constellation mapping module of the SC-FDMA Autoencoder.

A. THE OPTIMIZATION OF POWER CONTROL
The power control unit of SC-FDMA autoencoder is shown as Fig .3. The goal of the power control unit is to find the optimal transmit power for each symbol. After an Mdimensional one-hot vector s in is fed into the power control unit, the transformation P = p (s in ) is applied to generate the output P of the power control unit. The function of the VOLUME 4, 2016 power control unit can be expressed as where W q and b q are the parameters of the q-th linear fully connected layer, and σ(u) = max(0, u) is the function of the ReLU activation layer [22]. In the backpropagation process for optimizing the trainable power control unit, assuming the loss of the SC-FDMA Autoencoder is l, then the gradient of the loss to the power P can be expressed as where N batch is the size of each training batch; s 3 (i) and s 4 (i) respectively denote the value of s 3 (t) and s 4 (t) of the i-th sample in the training batch; ∂l ∂s4(i) is the gradient of the loss to s 4 (i), and it is determined by the operation function of the following modules and the loss function of the SC-FDMA Autoencoder. After the gradient of loss to P is calculated, the transmit power P can be optimized by using the adaptive moment estimation (Adam) optimization method [21]. The optimization steps of the transmit power P can be expressed as follows: where χ τ and v τ are the first-order momentum term and the second-order momentum term at iteration τ , respectively; the hyperparameters β 1 and β 2 are the dynamic values of the first-order momentum term and the second-order momentum term, respectively;χ τ andv τ are the estimated value of firstorder momentum term and the second-order momentum term at iteration τ , respectively; 0 is a very small number added to prevent the denominator from being 0; lr is the learning rate of the optimizer, and P τ is the trainable transmit power at iteration τ .

B. THE OPTIMIZATION OF MODULATION
The constellation mapping module and the receiver neural network of the SC-FDMA Autoencoder is shown in Fig. 4. The goal of the constellation mapping module is to find a two-dimensional representation of the input M -dimensional one-hot vector. The value of each dimension of the twodimensional output refers to the in-phase and quadrature component respectively. After an M -dimensional one-hot vector s in is fed into the constellation mapping module, the transformation s out = f (s in ) is applied to generate the output signal s out . It should be noted that the power of s out is normalized to 1. The function of the constellation mapping module can be expressed as where Norm (u i ) is the function of the batch normalization layer [23], and it can be expressed as where γ and δ are parameters to be learned, u i is the ith output of the ReLU layer. The receiver neural network operates similarly to the constellation mapping module. It applies the transformation r 3 = g (r 2 ) to generate the output signal r 3 of the receiver neural network. The function of the receiver neural network can be expressed as The cross-entropy loss is used in the training of the SC-FDMA Autoencoder. First, the Softmax function is operated on the output of autoencoder r 3 to generate the probability distribution of the output belonging to each input symbol: where p i,j is the prediction probability that the real category of the i-th transmitted signal is equal to j. Then, the loss and the parameters of the autoencoder are calculated. The loss function can be expressed as where R i is the i-th row of r 3 ; R i (j) is the value of the j-th column of R i ; y i,j is a symbol-function. If the real category of the i-th transmitted signal is equal to j, y i,j takes 1; otherwise, it takes 0.

IV. SIMULATIONS AND DISCUSSIONS
In this section, the simulation results of the proposed method are provided. In the simulation, the mapping pattern of SC-FDMA is localized mapping; the size of frequency-domain data is 32, and the number of subcarriers is 256. Besides, the size of the training set is 51200; the batch size is 256; the initial learning rate is 0.001, and it decreases by 20% every 5 epochs. The training is stopped when the loss does not drop for 10 epochs.
In the simulations, for convenience, the output signal of PA is calculated by the parameters and the behavioral model  of PA. The behavioral model of the PA performs odd-order Taylor series expansion of the Saleh model, and it has the form [24]: where x in and x out are the PA input and output magnitude, respectively; α n is the n-th coefficient of the odd-order Taylor series, and it can be expressed as [25]: where G is the power gain in dB, and IP n is the output norder intercept point in dBw. For a practical PA module ZRL-2400LN, G is 28 dB, and IP 3 is 15 dBw. Substituting these values into (20) and (21), we have α 1 = 17.8 and α 3 = −118.6. For ZRL-2400LN+, another PA module with more VOLUME 4, 2016 severe nonlinearity, G is 28 dB, and IP 3 is 12 dBw. Based on this mode, we have α 1 = 17.8, α 3 = −666.7. First, the SER and the transmit power of the proposed method are compared with those of the PBO method with 64square QAM (64-SQAM), which is a square shaped constellation. In Fig. 5, the peak transmit power is decreased to the compression point of 1 dB to make the PA operate at a linear region. When the four curves have an identical SER of 10 −2 , the proposed method achieved a gain of 3.9 dB in the noise power compared with PBO for ZRL-2400LN, and the gain increases to 4.6 dB for ZRL-2400LN+. Therefore, it can be seen that the proposed method can achieve more performance improvement for the PA under more serious nonlinearity. The ratio of the average power of PA to the saturated power of PA is shown in Fig. 6, it can be found that the proposed method enables the PA to operate at a higher power than the PBO method, and the power improvement is greater for ZRL-2400LN+, which has a more serious nonlinearity. This is because the PBO method reduces more transmit power to make a more nonlinear PA operate at a linear region. Therefore, the gain of the proposed method should be greater when the nonlinearity of PA is more serious. Besides, Fig. 5 and Fig. 6 also demonstrate that the reduction of SER by the proposed method is attributed to the optimization of transmit power.
Next, the influence of the modulation order M on the performance is analyzed for ZRL-2400LN.     Table. 1, for M = 16, 64, the PAPR of the proposed constellation is lower than 16-SQAM and 64-SQAM, which makes the PBO for 16-SQAM and 64-SQAM reduce more power than PBO for QPSK. Furthermore, the proposed method is compared with the DPD method of M -SQAM and polygon constellation [26] with traversed power. In the comparison, the transmit power P is traversed from 0 to the saturated power P sat at an interval of 10 −3 to find the transmit power that leads to the lowest SER. The optimized power of the comparison method can be expressed as where d m is the m-th signal of the M possible transmitted signals with normalized power; f m is the received signal when d m is transmitted; Z m is the decision region of the transmitted signal d m . The SER of the proposed method and the DPD method of 64-SQAM and 64-order polygon constellation with traversed power is shown in Fig. 9.
It can be found that although the proposed method and the DPD method of 64-SQAM and 64-order polygon constellation have an identical SER of 10 −2 , the DPD method operates  at the traversed power that minimizes the SER. Compared with the DPD method of 64-order polygon constellation, the proposed method achieves a gain of about 1.5 dB in noise power, and the DPD method of 64-SQAM achieves a gain of about 1.3 dB. For these given constellations, the DPD method obtains a lower SER performance than the proposed method at any transmit power, which demonstrates the benefits of the joint optimization of constellation shape and transmit power.
In Fig. 10, the SER of the proposed method is compared with that of the linear PA with the same modulation and PA output power under M = 16, 64. It can be seen from Fig.  10 that while the proposed method and the linear PA have an identical SER of 10 −2 , the proposed method is 0.1 dB worse than the linear PA in noise power when M = 16, and 0.15 dB worse than linear PA in noise power when M = 64. The SER lower bound of nonlinear PA cannot be even lower than that of linear PA, and the SER of nonlinear PA with the proposed method is close to the SER of linear PA with the same modulation. Thus, the SER of the proposed method is close to the SER lower bound of nonlinear PA.

V. CONCLUSION
In this paper, a system design for a nonlinearity-constrained system is proposed. By training the constellation mapping module, the power control unit, and the receiver neural network of the SC-FDMA Autoencoder, the constellation shape, transmit power, and the demodulation scheme can be jointly optimized to minimize the SER for the given channel and the PA in the SC-FDMA system. This can increase the  transmit power and make the PA operate in a region of severe nonlinearity. Based on this, the high-power transmission performance can be improved for the scenarios such as satellite communication and communication systems with severe interference. The simulation results indicate that the proposed method outperforms conventional methods in terms of SER and transmit power, and the SER of the proposed method is close to the SER lower bound of nonlinear PA.