A Learning-Based End-to-End Wireless Communication System Utilizing a Deep Neural Network Channel Module

The existing end-to-end (E2E) wireless communication systems require fewer communication modules and have a simple processing signal flow, compared to conventional wireless communication systems. However, in the absence of a differentiable channel model, it is impossible to train transmitters, used in such systems, which makes impossible achieving optimal system performance. To solve this problem, E2E wireless communication systems, learned with conditional generative adversarial networks (CGANs) for channel modeling, have been proposed recently. Unfortunately, the CGAN training is prone to instability, slow convergence, and inaccurate channel modeling, which affects the system performance. To this end, a learning-based E2E wireless communication system, utilizing a deep neural network (DNN) channel module to model an unknown channel, is proposed in this paper. Simulation results show that the proposed DNN channel modeling has faster convergence, simpler network structure, and can reflect the behavior of real channels more accurately. In addition, the proposed learning-based E2E wireless communication system performs better, in terms of the bit error rate (BER) and block error rate (BLER), than the learning-based E2E wireless communication system, using CGAN as unknown channel, and a traditional communication system, designed based on the prior knowledge of the channel. Compared to these two systems, at high signal-to-noise ratio (SNR) values, the proposed system can achieve a SNR gain of at least 2 dB, in communication scenarios involving frequency-selective multi-path channels.

The conventional wireless communication system architecture consists of multiple signal processing sub-modules, shown in Fig. 1. Message s, which must be sent to the other side, is transformed by the transmitter into a suitable (encoded and modulated) signal x, which after passing through the channel, becomes signal y, which arrives at the receiver and is converted into messageŝ by performing opposite procedures to that of the transmitter. The received messageŝ is generally (slightly) different from the originally sent message s due to errors occurring in it, caused by the interference and noise existing in the channel. The modules of the transmitter and receiver, performing various transformation operations over data and signals, are individually optimized for specific channel environments, but the optimization of communication sub-modules cannot guarantee the global optimum of the whole communication system, [6].
Over the past few decades, deep learning (DL) techniques made impressive developments in fields such as computer vision and natural language processing (NLP), demonstrating excellent performance. However, due to the considerable complexity of the physical layer and the need for realtime communication in many cases, application of DL technology in the field of communications develops slowly. Recently, due to the significant development of computer hardware, the computational speed has increased dramatically and the time to train models using DL techniques has decreased dramatically. Thus, these techniques have been widely applied in the field of communications recently, e.g., to improve the performance of conventional communication modules, including modulation identification [7], [8], channel estimation [9], [10], channel equalization [11], [12], etc., or to develop DL-based end-to-end (E2E) wireless communication systems [13]. Such systems generally consist of a transmitter, a channel model, and a receiver, whereby both the transmitter and receiver can be represented as a deep neural network (DNN) and the whole system can be interpreted as an autoencoder [14]. The main objective of designing E2E wireless communication systems is to maximize their performance by optimizing the transmitter and receiver operation.
E2E wireless communication systems represent a very attractive idea in theory but have the disadvantage of requiring a differentiable channel model when implemented in practice. When the channel model is not differentiable, it is not possible to train the transmitter. The channel transfer function can be assumed, of course, but the actual wireless channel characteristics are affected by factors such as operating frequency and propagation environment, and are constantly changing. So, the assumed channel transfer function reflects somewhat differently the actual channel characteristics. An E2E wireless communication system, trained using an assumed channel transfer function, is not an optimal system. In addition, the DNN structures utilized by the current E2E wireless communication systems are complex, and the training process is prone to instability and slow convergence, so it is necessary to develop an E2E wireless communication system with a simple structure and easy model training.
In this paper, we propose a learning-based E2E wireless communication system, based on a DNN channel module, to solve the above-mentioned problems. First, the data generated by the transmitter are passed through the DNN channel module and separately to a statistical channel model. The simulation data outputted by the DNN channel module are compared with the data outputted by the statistical channel model, and the loss function between them is optimized using the Adam optimizer [15]. The data outputted by the optimized DNN channel module can infinitely approximate the data outputted by the statistical channel model. Compared with conventional channel modeling approaches, the proposed DNN channel module can learn stochastic channel behavior directly from the data without the need for expertise. Furthermore, when the transmitter in the E2E wireless communication system is trained, the already optimized DNN channel module can serve as a bridge for back-propagation of gradients, which allows the parameters of the transmitter to be updated. If a provision for such back-propagation is not made, only the receiver can be optimized, and it is not possible to reach optimal performance of the E2E wireless communication system because this requires optimization of both the transmitter and receiver. Compared with existing E2E wireless communication systems, the proposed system is simple in structure, fast in convergence, and applicable to complex communication scenarios which are difficult to describe by simple mathematical models.
The main contributions of the paper can be summarized as follows: For the cases when the realistic stochastic channel response is unknown or not easily modeled by a mathematical form of analytics, a DNN channel module is proposed to model the actual channel distribution. The trained DNN channel module can generate signals that are more similar to the real signals and thus respond more accurately to the stochastic channel behavior. It is demonstrated that the trained DNN channel module can be used as a bridge for the gradient backpropagation instead of the real channel to update the transmitter parameters, which is useful in the case of an unknown channel model, which otherwise would prevent the learning of the E2E system by blocking the gradient back-propagation from the receiver DNN to the transmitter DNN. By means of simulations it is proven that the proposed channel modeling with the designed DNN channel module converges faster and is more stable compared to the conditional generative adversarial networks (CGANs) channel modeling. In addition, the proposed learningbased E2E wireless communication system, utilizing the DNN channel module, performs better than the E2E wireless communication system with CGAN as unknown channel, in terms of both the bit error rate (BER) and block error rate (BLER). The rest of this paper is organized as follows. The background and related work in the area are presented in Section II. The training procedure of the E2E wireless communication system, utilizing a DNN channel module, is described in Section III. Experiments and simulation results obtained are presented in Section IV. Finally, Section V concludes the paper. Table 1 contains the list of main abbreviations used throughout the paper.

II. BACKGROUND AND RELATED WORK
This section presents the background of autoencoders and autoencoder-based E2E wireless communication system architectures, and concludes with the latest research progress in the area of E2E wireless communication systems.

A. AUTOENCODER BASICS
As shown in Fig. 2a, an autoencoder [14] is an unsupervised model that uses an artificial neural network (ANN) in learning. It is based on a back-propagation algorithm that uses the input data s as a supervision to guide the ANN to learn a mapping relationship to obtain a reconstructed outputŝ. The autoencoder model consists of two main parts, including an encoder function h = f (s) that converts the input s into the intermediate variable z, and a decoder functionŝ = g(h) that subsequently converts the intermediate variable z intoŝ. The main function of the autoencoder is to reconstruct the input s so thatŝ is infinitely close to s or even identical to it. The autoencoder has the advantages of high generalization ability and being unsupervised without data annotation. The learning process of the autoencoder can be described as minimizing the reconstruction loss: where L denotes the loss function, e.g., a cross-entropy loss function, a mean square error (MSE) loss function, a mean absolute error (MAE) loss function, etc., used to penalize g(f (s)) for being different from the input s.

B. AUTOENCODER-BASED END-TO-END WIRELESS COMMUNICATION SYSTEM
As shown in Fig. 2b, the architecture of a learning-based E2E wireless communication system consists of three main components, including a transmitter, a channel model, and a receiver. The main goal is to optimize the transmitter and receiver, represented by an encoder and decoder composed VOLUME 11, 2023 of DNN, respectively. As shown in Fig. 1, in conventional communication systems, the transmitter can be decomposed into modules such as a source encoder, a channel encoder, and a modulator, and respectively, the receiver can be decomposed into modules such as a demodulator, a channel decoder, and a source decoder. As shown in Fig. 2a, a regular autoencoder, usually compresses the input data s to obtain the low-dimensional data z and reconstructŝ at the output with minimum error. However, encoders in learning-based E2E wireless communication systems usually add redundancy to the input data s to obtain x as to enhance the immunity of the transmitter's output signal x to noise, interference, and various uncertainties in the channel.
In an E2E wireless communication system, the transmitter wants to communicate one out of M = 2 k possible messages s to the receiver, making n discrete uses of the channel. Each message consists of n bits, k of which are information bits, and the rest (n − k) are redundant bits, e.g., produced by channel encoding. The communication rate of such communication system is R = k/n [bit/channel use] (in this paper, a communication rate of 0.5 is used). The source data s ∈ R p are passed through the encoder function x = f (s, θ f ) to obtain the output vectorŝ ∈ R q of the transmitter. Usually, the transmitter imposes certain constraints on x, such as energy constraint ||x|| 2 2 ≤ n and amplitude constraint |x i | ≤ 1∀i [10]. In this paper, we impose constraint E |x i | 2 ≤ 1∀i on the average power of x. The transmitted signal x goes through the wireless channel, containing noise and interference. The output vector y of the channel is passed through the decoder functionŝ = g(y, θ g ) to obtain the outputŝ of the receiver, where θ f and θ g denote the trainable parameters of the encoder and decoder, respectively. These trainable parameters can be updated by optimizing the loss function L(s,ŝ) using the stochastic gradient descent (SGD) algorithm, and thus an E2E wireless communication system with optimal performance can be obtained.

C. RESEARCH PROGRESS OF END-TO-END WIRELESS COMMUNICATION SYSTEMS
An E2E wireless communication system was first proposed in [13], which does not require complex conventional communication modules, and the whole system is entirely composed of DNN. As a purely data-driven approach, it has attracted wide attention from scholars and has been studied in several other works [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33]. However, in the E2E wireless communication systems proposed in [13], [16], and [17], the input information is represented by one-hot vectors, which have the disadvantage of carrying less information compared to random sequences of the same length. In the E2E wireless communication systems presented in [18], [19], and [20], even though binary sequences are used instead of one-hot vectors, these are useful for small data blocks only, which is not practical for real communication. To address this issue, the E2E wireless communication systems in [21] and [22] use a convolutional neural network (CNN), which allows the length of the input sequence to be further extended.
In the communication systems designed in [13] and [16], the channel is represented by the conditional probability density function p(y|x). However, in the actual implementation, the channel model, or more precisely, the gradient of the channel transfer function, must be known so that an E2E optimization can be done. Because the channel is considered as a black box during the communication, one can only observe the input and output of the channel, which prevents the back-propagation of the gradient to train the transmitter. To address this problem, reinforcement learning algorithms are used in [18] to overcome the drawbacks of autoencoderbased E2E wireless communication systems that require differentiable channel models. In [23], the authors introduce the transfer learning idea to solve this problem. In [24], a new alternating algorithm is proposed to solve the channel gradient loss problem. In [25], the authors model the wireless channel using untrainable stochastic convolutional layer. CGAN is used in [22] and [26] to approximate the wireless channel response. However, the CGAN model is difficult to train and is prone to training instability, slow convergence, and gradient disappearance, which can lead to performance degradation of the E2E wireless communication system. To alleviate the gradient disappearance problem, in [27], the authors propose a residual aided GAN (RA-GAN)based training scheme. However, this scheme increases the complexity of the network and is computationally intensive.
To solve the above-mentioned problems, we propose an E2E wireless communication system utilizing a DNN channel module, in which both the transmitter and receiver consist of a one-dimensional CNN that can input infinitelength sequences. Used for channel modeling, the proposed DNN channel module can avoid the complex network structure and training instability of the CGAN-based model, which in addition requires two neural-network modules (i.e., a generator and a discriminator) to be trained alternately. The simple neural-network structure, utilized by the proposed learning-based E2E wireless communication system, allows to greatly reduce the number of trainable parameters and lower the computational effort. The number of parameters of the CGAN model is equal to 825,950, whereas the number of parameters of the model proposed in this paper is less, i.e., equal to 677,013. Both models utilize a CNN structure, but the computational complexity of the proposed model is lower. Moreover, the proposed model only needs a single network module and a loss function to fit the output distribution of a real channel; it does not require two neural network modules for confrontational training like in the CGAN model. In addition, during the training process, CGAN is prone to non-convergence and model collapse problems, which the model proposed in this paper does not suffer from. In addition, its training process is more stable, and convergence is faster. Table 2 compares the complexity of these two models. Differently from the CGAN-based model, the model proposed in this paper does not simulate the output distribution of the statistical channel model, but keeps the output data of the DNN channel module the same as the output data of the statistical channel model in real-time, so that the output distribution of the DNN channel module to be naturally the same as that of the statistical channel model. Finally, the proposed model can directly optimize the E2E wireless communication system, based on data measured in a real environment.
In addition, in E2E wireless communication systems utilizing orthogonal frequency division multiplexing (OFDM) [28], [29], researchers have also investigated how to reduce the number of pilot frequencies and cyclic prefixes (CP) used, or even to ensure that BER does not increase when these are not used at all. An E2E learning framework has also been applied in the field of optical and molecular communications [30], [31], [32], [33], and even in the field of communication security [34]. Compared to conventional communication systems, autoencoder-based E2E wireless communication systems are more robust under complex channel conditions.

III. END-TO-END WIRELESS COMMUNICATION SYSTEM UTILIZING A DNN CHANNEL MODULE
In this section, the structure and training procedure of the proposed learning-based E2E wireless communication system, utilizing a DNN channel module, are described in detail.

A. SYSTEM OVERVIEW
As shown in Fig. 3, the proposed learning-based E2E wireless communication system, utilizing a DNN channel module, has the same structure as other E2E wireless communication systems, where the transmitter and receiver are represented by neural networks. During the training process, the DNN channel module, transmitter, and receiver are trained sequentially.
The value of the loss function between the transmitter and receiver can be minimized after several rounds of iterative training, which results in an optimal performance of the E2E wireless communication system.
The Huber loss function [35] is used in training the DNN channel module for channel modeling, as per (2), where a usually refers to the residual, i.e., the difference between the observed and predicted values a = y −ŷ. In the conducted experiments, y is the output value of the statistical channel model andŷ is the output value of the DNN channel module, so (2) can be extended to (3), and this loss function is optimized using the Adam optimizer. When the value of the loss function is minimized, a DNN channel module comparable to the statistical channel model is obtained.
The Huber loss function is a combination of MSE and MAE, and contains a hyperparameter δ, whose value determines the focus of the Huber loss function -on MSE or MAE. When |y −ŷ| > δ, it becomes similar to MAE and the gradient always approximates δ, which ensures that the model updates the parameters at a faster rate. When |y −ŷ| ≤ δ, it becomes MSE, and the gradient decreases gradually, which can ensure that the model gets the global optimum more accurately. Therefore, the Huber loss function has the advantages of both MSE and MAE. In addition, the problem of sensitivity to outliers is reduced, and the function of derivability everywhere is realized. The hyperparameter δ is set to 0.1 in the conducted experiments.
The binary cross-entropy loss function is used in training the transmitter and receiver. In the conducted experiments, the input is a random bit sequence s, which is passed through the transmitter and channel module to produce the output y, which is finally passed through a Sigmoid activation function in the last layer of the receiver to get the recovery information VOLUME 11, 2023 Y. An et al.: Learning-Based E2E Wireless Communication System Utilizing a Deep Neural Network Channel Modulê s. Therefore, the distance betweenŝ and s is measured using the binary cross-entropy loss function, which can be expressed as: where s n andŝ n denote the n th element of s andŝ, respectively, and s n ∈ {0, 1} andŝ n ∈ [0, 1] denote a probability value of 0 or 1. When using CGAN for channel modeling, the following two loss functions are used, respectively for the discriminator (D) and generator (G): D(G(z, m)))]; (5) where m represents additional information.
In the learning-based E2E wireless communication system proposed in this paper, the instantaneous channel state information (CSI), h, is a randomly selected sample from a large channel set. For time-varying channels, by directly using the received signal y and the received pilot frequency data y p as inputs, the receiver can automatically infer the channel condition and recover the transmitted data. In the proposed system, the use of the pilot symbols has two advantages, namely implicitly estimating the CSI and ensuring the best system performance. In the absence of the pilot frequency symbols, the performance of the E2E wireless communication system is significantly degraded, as demonstrated in [22].

B. TRAINING PROCEDURES
In the proposed learning-based E2E wireless communication system, the DNN channel module is first trained. The randomly generated binary data are passed through the transmitter to get the encoded data x, which are passed through the statistical channel model to get the received data y by the receiver. At the same time, x is made to pass through the proposed DNN channel module to getŷ. The distance betweenŷ and y is calculated using (3), and the weights of the DNN channel module are updated by optimizing the loss function. When the outputŷ of the optimized DNN channel module gets similar to the output y of the statistical channel model, the obtained DNN channel module can act as a bridge for gradient back-propagation when the transmitter is trained. After several iterations, theŷ output of the DNN channel module can infinitely approximate the y output of the statistical channel model.
Next, the transmitter is trained. For this, the above trained DNN channel module is used as an alternative channel. During the training process, the transmitter, the DNN channel module, and the receiver form a complete neural network structure, at which point the network parameters of the DNN channel module and of the receiver need to be fixed. If the network parameters of an untrained receiver are not fixed, then this training becomes joint training. In training the transmitter and receiver, the goal is to make the loss function value as small as possible. If the transmitter and receiver are jointly trained, one can make the value of the loss function small enough by adjusting the weight of the last layer of the receiver, but the transmitter may not be fully trained at all and, as a result, in practical applications the performance of the E2E wireless communication system cannot be fully utilized. A cross-entropy loss function (4) is used at the output of the receiver to calculate the loss between the input data s and datâ s outputted by the receiver. The gradient can be propagated back to the transmitter through the DNN channel module, and the weights of the transmitter can be updated according to SGD.
Finally, the receiver is trained. The receiver, the statistical channel model, and the already trained transmitter form a complete neural network structure, at which point the network parameters of the transmitter need to be fixed. As the loss function is calculated at the output of the receiver, the receiver is easier to train than the transmitter. When the performance of the E2E wireless communication system is tested, the trained transmitter and receiver are used on the statistical channel model.

IV. EXPERIMENTS
In the conducted experiments, a Rayleigh fading channel and a frequency-selective multi-path channel were simulated first, using the designed DNN channel module. Then, the proposed learning-based E2E wireless communication system, utilizing this DNN channel module, was compared (in terms of BER and BLER) with the learning-based E2E wireless communication system that uses CGAN as unknown channel [22] and with a traditional OFDM system, designed based on the prior knowledge of the channel. The network parameters of the proposed learning-based E2E wireless communication system are shown in Table 3.
Randomly generated binary data were used as a training data set with parameter K = 200 * 64 (c.f. Table 3), meaning that each message s, sent by the transmitter, consists of 200 data blocks with size of 64 bits each. The testing data set for each SNR value contained 10,000 data blocks of 64bit length, with randomly generated bits. Furthermore, the experiments were performed on Windows AMD Ryzen 7 5800H CPU and NVIDIA GeForce RTX 3060 GPU.
The core of the proposed DNN channel module is a CNN. CNNs, in general, solve the problem of parameters expansion of neural networks composed of fully connected layers. In a typical CNN, not all neurons in the upper and lower layers are connected directly, but rather through a ''convolution kernel'' acting as an intermediary, which allows to reduce the number of trainable parameters. The same ''convolution kernel'' is shared on one-dimensional (or twodimensional image) data, which can still retain the original positional relationship through convolution operations. CNN can theoretically process infinitely long one-dimensional data.

A. CHANNEL MODELING 1) RAYLEIGH FADING CHANNEL
The Rayleigh fading channel is a statistical model of a radio signal propagation environment. This model assumes that after passing through a radio channel, the amplitude of the signal is random, and its envelope obeys a Rayleigh distribution. This channel model enables the description of shortwave channels reflected by the ionosphere and troposphere, as well as urban environments with dense buildings, [36]. The channel output is determined by y n = h n · x n + w n , where h n ∼ CN(0, 1). The additive Gaussian white noise variance added to the channel is σ = (2RE b /N 0 ) −1 , where E b /N 0 indicates the ratio of the energy per bit (E b ) to the noise power spectrum density (N 0 ).
The main principle of the channel modeling proposed here is to train the DNN channel module by narrowing the gap between the statistical channel model's output data and the DNN channel module's output data. Therefore, under the premise that the structure of the DNN channel module remains unchanged, choosing an appropriate loss function can ensure the desired channel modeling effect. The loss function (3) is used for modeling with the DNN channel module. In modeling the Rayleigh channel, we compare the loss when the DNN channel module and CGAN are separately used for channel modeling, as shown in Fig. 4 (note that the axis scales of Fig. 4a and Fig. 4b are different). The reason why the losses shown in Fig. 4 have different orders of magnitude is that two different loss functions are used. The loss function (3) is used with the DNN channel module, with the main purpose to reduce the gap between the generated data and real data. In the process of CGAN training, the discriminator and generator work against each other, so when one improves, the other deteriorates, using loss functions (5) and (6), respectively. The task of the generator is to produce fake data that is indistinguishable from real data. The task of the discriminator is to distinguish real data from fake data. In the early stage of training, the generated fake data is far from the real data, so the discriminator has a better loss value. Since the task of the generator is difficult, it is initially hard to find a good gradient to follow during training, so the loss of the generator behaves somewhat stochastically in the early stages of training. When the generator starts to improve (after about 350 epochs), the task of the discriminator becomes more difficult, and its VOLUME 11, 2023 performance worsens. This is a good sign that the training scheme is working. Finally, when the CGAN training is stable, the discriminator cannot identify whether the data are real or fake, that is to say, discriminator has been reduced to invalid (the probability is 0.5 everywhere). The loss of the discriminator at this time is L(D) = ln(0.5) + ln(1 − 0.5) = 1.386, but for the generator the loss is L(G) = ln(1 − 0.5) = 0.693, which explains the different orders of magnitude of losses.
As can be seen from Fig. 4, the loss converges more quickly when modeling with the DNN channel module is used, in which case a steady state is reached after 800 epochs. However, when CGAN is used for channel modeling, though the Nash equilibrium is reached after 3500 epochs, training instability and slow convergence are observed. Therefore, the proposed DNN channel modeling is faster and easier to train, compared to the CGAN modeling. To verify the simulation capability of the DNN channel module for the Rayleigh fading channel model, we compared the output distribution of the Rayleigh fading channel with that of the DNN channel module, as shown in Fig. 5, illustrating that both are similar by observation. In order to measure the similarity between the output distribution of data generated by the DNN channel module, and the output distribution of the Rayleigh fading channel model, we used the Kullback-Leibler (KL) divergence. Basically, the closer two probability distributions are, the smaller the value of the KL divergence. We also compared the output distribution of data, generated by CGAN, with that of the Rayleigh fading channel model. Both results are shown in Fig. 6. As can be seen from the figure, the proposed DNN channel modeling can reach a very small KL value only after 500 epochs, whereas CGAN needs 1750 epochs to reach the same KL value. This proves that the data distribution generated by the proposed model is almost the same as the output distribution of the Rayleigh fading channel model, and the proposed model is faster and more stable than CGAN.

2) FREQUENCY-SELECTIVE MULTI-PATH CHANNEL
In general, multiple paths for wireless propagation of radio signals exist, each having different time of arrival at the receiver. If the relative time delay of multi-path signals is nonnegligible when compared with the time duration of a symbol, then there will be an inter-symbol interference (ISI) while multi-path signals are superimposed, and the corresponding channel is called frequency-selective multi-path channel. The impulse response of a complex baseband channel can be expressed as: where K p denotes the total number of paths, p(t) denotes the communication system's shaping pulse, and b k , θ k and τ k denote the path gain, phase shift, and time delay of the k th path, respectively. In the conducted experiments, a three-tap channel with equal average power was used as a frequencyselective multi-path channel. In order to further describe the rapid convergence of the proposed DNN channel model, Fig. 7 illustrates the loss of frequency-selective multi-path channel modeling based on the proposed DNN channel module and that of CGAN. As can be observed from the figure, the loss of channel modeling based on the DNN channel module converges after 1000 epochs. However, the CGAN-based model reaches Nash equilibrium only after 7000 epochs, but the loss functions still fluctuate significantly and the training is unstable. These results demonstrate again that the proposed model enjoys faster and more stable training.
For the frequency-selective multi-path channel, the data distribution output of the statistical model was compared with that of the proposed DNN channel module, as shown in Fig. 8. The horizontal axis represents the size of the output data with an interval of 0.5. The vertical axis represents the relative frequency of data appearing in intervals of 0.05. As can be seen from the figure, the output distribution of the DNN channel module is slightly higher in proportion near 0, but overall, the output distributions of the two models are similar. Therefore, the proposed DNN channel modeling can be applied in wireless communication scenarios involving complex channel distortion and interference.
To further demonstrate the feasibility of the proposed modeling approach, we calculated the KL divergence. The obtained results are shown in Fig. 9. As can be seen from the figure, the proposed DNN channel model can reach a very small KL value only after 250 epochs, whereas CGAN cannot reach stably the same KL value even after 2000 epochs. The final KL value, reached by the proposed DNN channel model, is close to 0, which again demonstrates that its output distribution is very close to that of the real channel. Therefore, when training the transmitter in an E2E wireless communication system, the already trained DNN channel modules can be used for gradient back-propagation. Moreover, the proposed model is faster and more stable than CGAN.

B. END-TO-END WIRELESS COMMUNICATION SYSTEM PERFORMANCE 1) RAYLEIGH FADING CHANNEL
As the Rayleigh channel is time-varying, the channel coefficient h n is constantly changing and thus unknown in the design of communication systems. Thus, CSI obtained by channel estimation should be used for the receiver to detect data transmitted. In traditional communication systems, it is required to estimate CSI explicitly and then use the estimated CSI to restore transmission symbols. In the method proposed in this paper, channel estimation can be completed implicitly through the DNN of the receiver according to the pilot symbols received, and the symbols transmitted can be directly restored at the output end of the receiver, with no need for expert knowledge.
In the conducted experiments, the proposed learning-based E2E wireless communication system, utilizing the designed DNN channel module, was compared with the learningbased E2E wireless communication system using CGAN as unknown channel [22] and with a traditional communication system, designed based on the prior knowledge of the Rayleigh fading channel. The traditional communication system consists of multiple signal processing modules, among which quadrature amplitude modulation (QAM) is used for modulation and recursive systematic convolution (RSC) codes with a code rate of 0.5 are used for channel coding, while the Viterbi algorithm is used for probabilistic decoding. For fair comparison, each block of the three communication systems considered includes 64 information bits.
In the two compared learning-based E2E wireless communication systems, the binary cross-entropy loss function and the Adam optimizer were used for training at a learning rate of 0.0001 with a fixed SNR of 15 dB. The performance of the compared communication systems was tested at SNR values ranging from 0 dB to 20 dB.
The obtained performance results of the three communication systems compared is depicted on Fig. 10.
The proposed learning-based E2E wireless communication system, utilizing a DNN channel module, performs slightly better than the other two communication systems, in terms of BER. In terms of BLER, the proposed system performs worse than the traditional communication system, but slightly better than the learning-based E2E wireless communication system with CGAN as unknown channel.

2) FREQUENCY-SELECTIVE MULTI-PATH CHANNEL
In other conducted experiments, the proposed learningbased E2E wireless communication system, utilizing a DNN channel module, was compared to the other two systems, based on a frequency-selective multi-path channel. The traditional OFDM system has 64 subcarriers, each of which is subject to 4-QAM modulation with a CP length set to 16 and RSC code with a code rate of 0.5 for channel coding. For fair comparison with this system, the proposed learning-based system and the learning-based E2E wireless communication system with CGAN as unknown channel were set to using 64 bits in block size with 16 zeros padding between two consecutive blocks. Setting of other parameters of the E2E wireless communication systems, based on the frequencyselective multi-path channel, was the same as that based on the Rayleigh fading channel.
As can be seen from Fig. 11, under frequency-selective multi-path channel conditions, the proposed learning-based E2E wireless communication system, utilizing a DNN channel module, performs significantly better than the traditional OFDM system in terms of BER. For 0 dB < SNR < 8 dB, the proposed system is similar in performance to the learning-based E2E wireless communication system with CGAN as unknown channel, but otherwise, for SNR > 8 dB, it outperforms the latter (the outperformance becomes significant for SNR > 17 dB). In terms of BLER, for 0 dB < SNR < 9 dB, the proposed system is comparable (but slightly better!) in performance to the other two systems, but otherwise, for SNR > 9 dB, it outperforms them (greater the SNR, greater the outperformance).
Overall, the proposed system shows better BER and BLER performance within the whole SNR range. The main reason for this is that the proposed DNN channel modeling can approximate the channel response and reflect the stochastic channel behavior more accurately. Thus, the features of the whole transmission sequence can be better utilized and extracted through the proposed learning-based E2E wireless communication system, utilizing the designed DNN channel module.

3) DISCUSSION OF RESULTS
The proposed channel modeling requires only a single DNN module, while the CGAN-based channel modeling [22] requires two DNN modules -a generator and a discriminator for repetition training. Therefore, the proposed DNN channel modeling is simpler and requires less computation effort, thus reducing the communication delay. The obtained experimental results show that the proposed modeling can solve the problems of the CGAN-based channel modeling related to the slow convergence, unstable training, and susceptibility to collapsing. Meanwhile, CGAN can reach Nash equilibrium only after several attempts made to set the network parameters. Besides, the proposed DNN channel modeling does not directly simulate the output distribution of the statistical channel model like the generator of CGAN, but rather keeps the data output by the DNN channel module being the same (in real time) as the data output of the statistical channel model. After a certain period of training, the obtained output distribution of the DNN channel module becomes naturally the same as the output distribution of the statistical channel model and thus more accurate than the output distribution obtained by direct simulation of the channel model.
As for the performance of E2E wireless communication systems under Rayleigh fading channel conditions, the proposed learning-based E2E wireless communication system, utilizing a DNN channel module, performs slightly better than the learning-based E2E wireless communication system with CGAN as unknown channel [22], in terms of BER and BLER. However, under more complex frequency-selective multi-path channel conditions, the proposed learning-based system performs significantly better than the learningbased E2E wireless communication system with CGAN as unknown channel, especially for greater SNR values. This is due to the fact that the channel model can be obtained more accurately through the proposed modeling method [ He has authored/coauthored more 100 research papers in refereed journals and conferences. His research interests include UCWW, the Internet of Things (IoT), cloud computing, big data management, and data mining.
IVAN GANCHEV (Senior Member, IEEE) received the Engineering (summa cum laude) and Ph.D. degrees from the Saint Petersburg University of Telecommunications, in 1989 and 1995, respectively. He is currently an International Telecommunications Union (ITU-T) Invited Expert and an Institution of Engineering and Technology (IET) Invited Lecturer associated with the University of Limerick, Ireland, the University of Plovdiv ''Paisii Hilendarski,'' and the Institute of Mathematics and Informatics, Bulgarian Academy of Sciences (IMI-BAS), Bulgaria. He was involved in more than 40 international and national research projects. He has authored/coauthored one monographic book, three textbooks, four edited books, and more than 300 research papers in refereed international journals, books, and conference proceedings. He has served on the TPC for more than 370 prestigious international conferences/symposia/workshops and as the guest editor for multiple international journals. He is on the editorial board of multiple renowned international journals. VOLUME 11, 2023