A Hybrid BP-EP-VMP Approach to Joint Channel Estimation and Decoding for FTN Signaling over Frequency Selective Fading Channels

This paper deals with low-complexity joint channel estimation and decoding for faster-than-Nyquist (FTN) signaling over frequency selective fading channels. The inter-symbol interference (ISI) imposed by FTN signaling and the frequency selective channel are intentionally separated to fully exploit the known structure of the FTN-induced ISI. Colored noise due to the faster sampling rate than that of the Nyquist signaling system is approximated by autoregressive process. A Forney style factor graph representation of the FTN system is developed and Gaussian message passing is performed on the graph. Expectation propagation (EP) is employed to approximate the message from channel decoder to Gaussian distribution. Since the inner product between FTN symbols and channel coefficients is infeasible by belief propagation (BP), we propose to perform variational message passing (VMP) on an equivalent soft node in factor graph to tackle this problem. Simulation results demonstrate that the proposed low-complexity hybrid BP-EP-VMP algorithm outperforms the existing methods in FTN system. Compared with the Nyquist counterpart, FTN signaling with the proposed algorithm is able to increase the transmission rate by over 40%, with only negligible BER performance loss.