Low-Complexity Detection Algorithms Applied to FTN-GFDM Systems

New applications for mobile networks are driving up data rates, requiring new solutions to meet the demand for throughput. In urban and densely populated areas, massive multiple-input multiple-output, high bands in millimeter waves, and ultra-dense networks based on small cells are being used to achieve high data rates. However, these solutions do not apply to the enhanced remote areas communications scenario, where large cells must be deployed using vacant VHF or UHF bands with an opportunistic spectrum allocation approach. In this case, a reasonable solution to increase the overall system data rate is to enhance the waveform spectral efficiency. This paper proposes a method to improve the spectral efficiency of the generalized frequency division multiplexing scheme by exploiting the faster-than-Nyquist principle in the time and frequency domains without introducing any penalties in terms of bit error rate. This achievement is obtained by under-sampling the time-frequency grid, which results in more data symbols being transmitted per waveform sample while introducing a controlled amount of interference into the transmitted waveform. On the receiver side, low complexity detectors, such as sphere decoder, successive symbol-by-symbol sequence estimation, successive symbol-by-symbol with go-back K sequence estimator, and frequency-domain equalization, can be used to reconstruct the transmitted data. This paper presents the details of adapting the mentioned detectors to the proposed waveform and analyzes their performance in terms of bit error rate under different channel models and implementation complexity. Among all the detectors considered in this paper, the sphere decoder stands out as it presents an interesting compromise between complexity and bit error rate, even in scenarios with high interference. Other detection techniques, originally proposed for multiple-input multiple-output systems, can be adapted to handle the interference introduced by the faster-than-Nyquist generalized frequency division multiplexing with reasonable complexity and acceptable bit error rate performance. Hence, one may conclude that the faster-than-Nyquist generalized frequency division multiplexing is a suitable candidate waveform for the enhanced remote areas communications scenario when high spectrum efficiency is required.


I. INTRODUCTION
The advent of the fifth-generation (5G) of mobile networks 27 is introducing applications that require high throughput and 28 spectrum efficiency [1]. Reduced coverage radius cells oper-29 ating at wide bandwidths above 20 GHz are being considered 30 The associate editor coordinating the review of this manuscript and approving it for publication was Pinjia Zhang .
to support these applications. 5G new radio (NR) [2], for 31 example, increases overall system efficiency by using mas-32 sive multiple-input multiple-output (MIMO) schemes oper-33 ating at a wide bandwidth in the millimeter-wave frequency 34 range and by using small cells in an ultra-dense network 35 (UDN). The latency reduction enforced by ultra-reliable low-36 latency communications (URLLC) applications also limits 37 the coverage area since the propagation time in this scenario is 38 VOLUME 10,2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ Over the years, several works have exploited the sequence 95 structure of FTN signaling to propose techniques based on 96 sequence estimation. In [7], the authors proposed a sequence 97 detector based on the sphere detector (SD). In this detector, 98 the search space is limited to a multidimensional sphere, 99 whose initial radius is defined by pre-estimate data provided 100 by a low-complexity detector. Moreover, it was shown in [7] 101 that when SD is applied to conventional binary phase-shift 102 keying (BPSK) FTN signaling under the additive white Gaus-103 sian noise (AWGN) channel, it can achieve near-optimal per-104 formance with reduced complexity. However, it is interesting 105 to observe that in the worst case, when the initial radius covers 106 all possible candidate sequences, the SD complexity can be as 107 high as that of the MLSE detector. In [8], the authors proposed 108 two low-complexity sequence detectors based on a successive 109 symbol-to-symbol interference cancellation algorithm. For 110 these detectors to work perfectly in a noise-free transmission, 111 an operating region is defined based on the prototype filter 112 and the FTN compression factor. In a noisy channel, the 113 successive symbol-by-symbol sequence estimator (SSSSE) 114 performance is degraded by error propagation. To solve this 115 problem, the authors proposed the successive symbol-by-116 symbol with go-back K sequence estimator (SSSgbKSE) 117 in which the previous K symbols are re-estimated to avoid 118 error propagation. In [9], the author proposed a sequence 119 detection technique based on frequency-domain equalization 120 (FDE). In this case, a cyclic prefix (CP) is inserted in each 121 transmission block, making the signal circular and allowing 122 low-complexity MMSE demodulation based on fast Fourier 123 transform (FFT) at the receiver. This proposal becomes inter-124 esting in scenarios where the block size is much larger than 125 the CP. In [10], the authors proposed an iterative detector 126 in the frequency domain based on the decision feedback 127 equalizer (DFE). The FTN receiver using iterative block DFE 128 (IB-DFE) has better BER performance than FDE. In contrast, 129 this type of receiver requires matrix inversions, and the com-130 putational complexity increases dramatically with the number 131 of iterations. With the exception of [11], which proposes 132 integrating FTN signaling into GFDM, these works consider 133 the conventional FTN system under the AWGN channel, with 134 compression in one domain only. 135 The main objective of this paper is to propose low compu-136 tational complexity detectors based on sequence estimation 137 to retrieve data bits in the FTN-GFDM receiver. The MLSE 138 is the optimal detector for the FTN-GFDM system but has 139 a prohibitive computational cost. The remainder of this paper is organized as follows.

158
In Section II, the FTN-GFDM system model is presented.
The number of subcarriers and subsymbols are defined as IfN M andN K are not integers, the squeezing factors must be 205 adjusted byv t =P/M andv f =S/K , and the GFDM 206 amplitude must be properly scaled, resulting in 207 Here, the constant v tvf normalizes the power of the FTN-210 GFDM signal. When v t < 1, intersymbol interference (ISI) 211 is intentionally introduced into the transmitted symbol. Sim-212 ilarly, when v f < 1, intercarrier interference (ICI) occurs.

223
Assuming that the FTN-GFDM signal is transmitted over a 224 multipath channel, the received vector after removing the CP 225 is given by where H is the circulant matrix based on the channel impulse 228 response h and w is the AWGN noise vector.

229
On the receiver side, a detector retrieves an estimate of the 230 transmitted data symbols by processing the received vector y. 231 Figure 1 depicts the simplified block diagram of the FTN-232 GFDM communication chain. After CP removal, the signal is 233 equalized with a simple ZF or FDE [13]. Then, a matched fil-234 ter (MF) is used to maximize the signal-to-noise ratio (SNR). 235 Although MF can extract some information from the received 236 symbol, it cannot handle the ISI and ICI introduced during the 237 modulation process. Therefore, a proper detection technique 238 must be used to retrieve the data symbols. Assuming the 239 perfect equalization of the channel, the discrete signal at the 240 MF output is given by  where . is the Euclidean norm and ξ N is the set of all 269 possible data symbol sequences.

270
The MLSE detector achieves good performance and can be 271 used to demonstrate that FTN-GFDM can increase spectrum 272 efficiency without sacrificing the symbol error rate (SER).

273
However, its complexity is prohibitive, and more feasible 274 detectors are needed for practical implementation.

276
Since MLSE is too complex for practical applications, this 277 paper considers SD [15] as an efficient method to solve (15). 278 The principle of the SD algorithm is to limit the search 279 space in a multidimensional sphere, seeking only candidate 280 solutions that are within the sphere of radius ρ, whose center 281 is the received vector r. Since the closest candidate within the 282 sphere is also the closest candidate in the entire search space, 283 SD achieves the optimal solution with lower complexity than 284 MLSE [16]. For a candidated to be within the sphere of 285 radius ρ, the following condition must be satisfied: Since the unconstrained maximum likelihood estimate is 288 ζ = (A H A) −1 A H r, the SD radius can be rewritten as Using the Cholesky decomposition [17], the matrix G H can 292 be represented by where L is an upper triangular matrix. Then, (16) can be 295 rewritten as The upper triangular matrix L allows the inequality (19) to be 298 expressed as Thus, the necessary condition for the N th data symbol to be 301 within the sphere is Since the SD algorithm is iterative, every d N that satisfies (21) 304 defines Therefore, the condition for the ith data symbol to be inside 310 the sphere is expressed by The algorithm can be seen as a search tree with a depth  (24) is not 319 satisfied, the node and all nodes below it must be discarded. The steps to implement the SD detector are summarized in 327 Algorithm 1.

328
The SD algorithm complexity is dictated by the partial 329 Euclidean distance computation ρ 2 i , line 4 in Algorithm 1.

330
Because of the upper triangular structure of L, the number of 331 operations required to compute ρ 2 i increases as i decreases.

332
Thus, the number of floating-point operations (FLOPs) [18] 333 required to calculate ρ 2 i in a node of level i is can be expressed by 344 Algorithm 1 SD Detector 1: Input: Upper triangular matrix L, unconstrained maximum likelihood estimate ζ , initial sphere radius ρ initial , modulation order J and constellation D 2: if i > 1 then 6: if ρ 2 ≥ ρ 2 i then 7: if ς i ≥ J then 10: while ς i > J do 12: while ς i > J do 24: Assuming the best-case scenario where only a single path 349 is traversed from the root to the first leaf node and the 350 remaining nodes are discarded, the lower bound for the SD 351 complexity in terms of FLOPs is given by  sample r n in a noise-free scenario can be expressed as where term a represents the ISI and ICI of the previous n 390 symbols, term b is the desired symbol, and term c represents 391 the ISI and ICI of the subsequent N − n − 1 symbols. To 392 estimate d n , one must remove the interference caused by the 393 other symbols. Thus, the interference caused by the n previ-394 ous symbols, already estimated, and by the N − n − 1 future 395 symbols, not estimated yet, must be removed. As proposed 396 in [8], the condition for perfect estimation for noise-free 397 transmission considering quadrature amplitude modulation 398 (QAM) modulation can be described by  to G 0,0 d n . Since G has complex elements in the FTN-GFDM 411 system, the estimation condition is given by term a represents the ISI and ICI from the previous n symbols. 425 It is important to note that SSSSE suffers from the effect of 426 error propagation. When a data symbol is incorrectly esti-427 mated, the estimation of subsequent data symbols suffers with 428 a stronger effect on neighboring symbols.

429
The steps to implement the SSSSE detector can be seen in

438
This means that the SSSSE complexity is lower compared 439 to the SD detector proposed previously in Section III.

441
SSSgbKSE is a symbol-by-symbol detection technique pro-442 posed in [8] to mitigate the influence of error propagation 443 that occurs in the SSSSE detector and increase the estima-444 tion accuracy. The idea behind SSSgbKSE is to increase the 445 estimation accuracy of the previous K symbols based on the 446 knowledge of the desired symbold n and then re-estimate 447 d n considering the improved estimates of the K previous 448 symbols. Consequently, (33) can be rewritten as where term a represents the ISI and ICI from the previous n 452 symbols, term b is the previous K symbols to be re-estimated, 453 term c is the desired data symbol to be re-estimated, and 454 term d is the ISI and ICI from the subsequent N − n − 1 455 symbols.

456
Similar to SSSSE, for satisfactory performance of 457 SSSgbKSE, it is necessary to satisfy the estimation condition. 458 First, the desired data symbol d n is estimated using the SSSSE 459 algorithm, and then, based ond n , the previous K data symbols 460 must be updated. For this purpose,d n−K is re-estimated by 461 where term a is the ISI and ICI from the previous n − K data 464 symbols and term b is the ISI and ICI from the subsequent K 465 data symbols.

466
After the previous K symbols have been re-estimated 467 according to (39), the desired data symbol d n must be 468 re-estimated as follows: Therefore, the SSSgbKSE detector has a higher complexity 493 than the SSSSE detector described in Section IV.

495
The FDE receiver was proposed in [9] as a solution for 496 the FTN system with low computational cost. In the pro-497 posed FDE-assisted structure, a short CP is added in each 498 transmission block, which enables low-complexity MMSE 499 demodulation based on FFT on the receive side [9]. Figure 4 500 shows the block diagram of this scheme.

501
Since the matrix G has a circulating structure, the eigen-502 values decomposition is given by Since W is a diagonal matrix, the estimator operates 519 symbol-by-symbol [13]. Thus, the estimated symbol in the 520 time domain is given by In addition, to facilitate the detection of the FTN signal, 523 the diagonal structure of the MMSE weight matrix reduces 524 the complexity required to calculate the inverse in (45) and 525 allows symbol-by-symbol detection. Thus, the computational 526 complexity of MMSE-based FDE is N log N [9], [13]. How-527 ever, the weight matrix proposed in [9] ignores the effects of 528 FTN noise correlation.

529
Under theses considerations and after the analysis of the 530 weight matrix, it is possible to conclude that the eigenvalues 531 of G play an important role in the performance of FDE. 532 Depending on the values of v t , v f , N , and the pulse shape, 533 the G matrix may contain eigenvalues close to zero, which 534 makes it difficult to detect the associated symbols, causing 535 BER penalties [13], [19] [20]. Therefore, the FDE detector is 536 expected to underperform in real FTN-GFDM applications. 537

538
The BER performance of the detectors described in the 539 previous sections is evaluated under AWGN, time-invariant 540 frequency-selective (TIFS), and time-variant frequency-541 selective (TVFS) channels. Table 1 shows the channel param-542 eters used in the simulations. The TVFS channel has four 543  The GFDM framework makes it easy to assign resource  SSSgbKSE achieves BER performance very close to that of 573 the optimal detector for values of K close to N . However, 574 SSSSE exhibits a performance loss of about 5 dB for the 575 AWGN channel, 4.3 dB for the TIFS channel, and 0.8 dB 576 for the TVFS channel compared to the MLSE detector. For 577 the parameters shown in Table 2, the G matrix is circulating 578 and, therefore, the FDE detector can be used. However, in 579 Fig. 5 (a), its BER performance is severely affected by (N − 580 N ) eigenvalues close to zero, resulting in a loss of about 581 5.8 dB for the AWGN channel in the reference BER. 582 Fig. 6 shows the BER performance for the frequency-583 compressed FTN-GFDM assuming the channels proposed 584 in Table 1: (a) AWGN, (b) TIFS, and (c) TVFS. Since the 585 real compression factorv f =S K = 5 6 = 0.8333 is 586 slightly higher than the compression factor in Fig. 5, the 587 SSSSE and SSSgbKSE detectors should perform better than 588 in the time squeezing scenario. In this case, the data rate 589 has increased by 20%. While the estimation condition is 590 maintained, SSSgbKSE can achieve performance very close 591 to that of SD, even with low values of K. The SSSSE detector 592 achieved a BER performance loss of approximately 3.5 dB 593 for the AWGN channel, 2.4 dB for the TIFS channel, and 594 0.5 dB for the TVFS channel, in relation to the optimal 595 detector. In Fig. 6 (a), it can also be seen that FDE has worse 596 performance than in Fig. 5 (a). For the AWGN channel, the 597 FDE detector achieves a BER performance loss of approxi-598 mately 7.5 dB. This is because the FTN-GFDM matrix for the 599 parameters presented in Table 2 is not circulating. Therefore, 600 this detector should not be used in this case. respectively. In this case, a root-raised cosine (RRC) proto-605 type filter with a roll-off factor of β = 0.5 was used. The 606 SD performance shows that it is possible to increase the data 607 rate by 50% without BER performance degradation using 608 v f = 0.75 and v t = 0.8. The squeezing factors v t and v f have 609 independent gains that can be combined to achieve higher 610 spectral efficiency. Since the estimation condition is violated 611 in this scenario, the SSSSE and SSSgbKSE detectors perform 612 poorly, while SD performance is still close to MLSE.    These results were obtained using the frequency squeeze 628 parameters shown in Table 2  The complexity statistics for MLSE, SD, SSSSE, and 636 SSSgbKSE detectors at different SNRs are shown in Table 3. 637 The number of FLOPs is calculated for the FTN-GFDM 638    Table 4 shows the big-O complexities of the detectors stud-652 ied in the previous sections of this paper. Note that the com-653 plexities are in descending order, i.e., from highest to lowest 654 complexity. The SSSSE complexity is determined only by N , 655 the SSSgbKSE complexity by N and K, and the MLSE com-656 plexity by N and J . Considering the asymptotic notation, the 657 MLSE has higher complexity, followed by the upper bound 658 of the SD, i.e., the worst-case scenario. Both detectors have 659 exponential complexity. However, in the lower bound, i.e., the 660 best-case scenario, the SD has quadratic complexity close to 661 that of SSSSE. For the SSSgbKSE detector, the worst-case 662 scenario is when K = N and the complexity becomes 663 cubic. However, for K = 1, the complexity is quadratic, 664 and the BER performance approaches that of SSSSE. Finally, 665 the FDE detector has logarithmic complexity and, therefore, 666 lower computational cost. However, as mentioned earlier, 667 its performance is worse than the other detectors, and its 668 applicability in the proposed FTN-GFDM system is limited. 669

670
Channel coding is a tool to improve the reliability and perfor-671 mance of wireless communication systems. Channel coding 672 aims to correct transmission errors through error detection 673 and error correction codes. These codes must have good error 674 correction capability, high flexibility, and low complexity. 675 Polar codes are flexible linear block codes with a low com-676 putational cost that can achieve channel capacity in discrete 677 memoryless channels. The polar code family was introduced 678 by Arikan [22], [23] and is based on channel polarization to 679 transmit information bits on reliable channels and frozen bits 680 on noisy channels. The modest coding and decoding com-681 plexity make these codes attractive for many applications. 682 Moreover, polar codes and low-density parity-check (LDPC) 683 were compared in [24], and it was found that polar codes offer 684 higher flexibility in terms of code rate compared to LDPC 685  channels. The polar code uses a coding rate of 1/2, a block 705 size of 2048 with 8 bits shortened, and successive cancellation 706 decoding. The BER performance is evaluated for hard-output 707 SD, SSSSE, and SSSgbKSE detectors over the channels in 708 Table 1. The performance gains achieved by the detectors in 709 the coded system with frequency compression can also be 710 seen in Table 5.

711
In addition, Fig. 11 presents the BER results for the 712 FTN-GFDM encoded with time-frequency compression over 713 the channel models in Table 1: (a) AWGN, (b) TIFS, and 714 (c) TVFS. In this scenario, the polar code also employs cod-715 ing rate 1/2 and successive cancellation decoding, but uses 716 blocks of size 1024 and 4 bits shortened. As expected, polar 717 coding increases the BER performance. The performance 718 gains achieved by the coding scenario for each detector are 719 shown in Table 5. 720 From the results presented in Figs. 9, 10, 11, and Table 5, 721 we can conclude that polar coding can provide performance 722 and reliability gains even in high ISI and ICI scenarios. 723  Moreover, SD continues to outperform the other detectors 724 analyzed, but at the cost of higher complexity.

726
Higher spectrum efficiency will be mandatory for all appli-727 cation scenarios envisioned for future mobile networks, and 728 increasing the data rate using efficient waveforms will be nec-729 essary where UDNs and massive MIMO cannot be employed.

730
FTN-GFDM is a very flexible waveform that can increase the 731 data rate by compressing the subsymbols and/or subcarriers.

732
However, the detection of the received signal is not trivial.

733
MLSE can retrieve the data symbols without imposing any 734 BER performance loss, but its high complexity hinders its 735 application in real-world conditions. In this paper, the adapta- mainly when the interference matrix is not circulant. The 748 SSSSE receiver has the lowest computational cost among the 749 nonlinear receivers, but its performance is suboptimal and 750 depends on the estimation condition. SSSgbKSE can perform 751 very closely to MLSE, as long as the operating condition is 752 respected and K is large enough. Thus, this technique can 753 be an interesting solution when the system parameters lead 754 to appropriate estimation conditions and the SD complexity 755 cannot be afforded. Nevertheless, the results evaluated in 756 this paper show that FTN-GFDM can significantly increase 757 the spectral efficiency of the system with controllable BER 758 performance degradation and reasonable complexity, indicat-759 ing that this waveform is an interesting candidate for future 760 mobile communication scenarios for which other spectrally 761 efficient techniques are not suitable.

762
Future research includes extending other detectors, such 763 as IB-DFE and message passing algorithm (MPA), to non-764 orthogonal waveforms with time and frequency compression. 765 Furthermore, future work should investigate the performance 766 gain of soft-output detectors in FTN-GFDM coded scenarios 767 and tree-search algorithms to find the log-likelihood ratios 768 (LLRs).