Nonlinear States-Truncated BCJR Equalization With States and Complexity Reduction for 100-Gaud PAM-8 IM/DD Transmission

In high-speed optical communication systems, inter-symbol interferences (ISIs) and nonlinear distortions can occur due to the bandwidth limitations and nonlinear responses of optical and electrical components at the transceiver. To address these issues, this paper proposes the use of Bahl, Cocke, Jelinek and Raviv (BCJR) algorithm based on nonlinear channel models, such as the Volterra nonlinear filter and lookup table (LUT). Additionally, we introduce an ultra-low complexity BCJR algorithm, referred to as states-truncated BCJR based on reduced-size LUT (RLUT-ST-BCJR), to mitigate the impact of bandwidth limitations and nonlinear distortions. The RLUT-ST-BCJR uses a pruned LUT as the channel model emulator to assist in the calculation of the transition metric. Only the states with probabilities greater than a set threshold are retained in the decoding process, resulting in significantly reduced computational complexity. The experimental results confirm the improvement of both BER and normalized general mutual information (NGMI) performance using the proposed RLUT-ST-BCJR algorithm, compared to the conventional linear BCJR equalization. With this approach, we successfully transmit a 100-GBaud PAM-8 signal in both back-to-back and 1-km fiber cases, under the BER threshold of 2.4E-2 assuming 20% overhead soft-decision forward error correction.