Iterative Receiver Design for Probabilistic Constellation Shaping in ISI Channel

This paper investigates the receiver design for probabilistic constellation shaping signaling over inter-symbol interference channel. The key component performing the constellation shaping is an adjustable distribution matcher, and the probabilistic shaping system is capable to adapt to variable data rates by adjusting the distribution match rate rather than the modulation or coding mode. In this paper, we resort to distinct techniques to derive two iterative receivers operating in time domain. Shaped-BCJR is a trellis-based solution where forward/backward algorithm together with turbo iteration is used to compute the posteriori probability in which the nonuniform a priori symbol probability initializes the calculation. Another receiver is based on linear filtering where expectation propagation provides a Gaussian approximation of the posteriori probability of each symbol. This receiver structure is represented by a factor graph and we detail the derivation of messages exchanging between adjacent nodes. The Bayesian equalization in each EP iteration brings unacceptable computational burden. An efficient block-wise matrix inversion strategy is proposed to tackle this problem, significantly reducing the computational complexity with little performance loss. Simulation results show that the proposed algorithms remarkably outperform LMMSE solutions and traditional EP based algorithms. Proposed matrix inversion strategy can also be used to improve the performance of other filter-type solutions.