Suppressed Noise Enhancement Equalization Combined With Nonlinear-MLSE Techniques for Bandwidth-Limitation IM/DD System

Bandwidth limitation introduced inter-symbol interference (ISI) is the major impediment for high baud rate intensity modulation and direct detection (IM/DD) system with commercial devices. Least mean square (LMS) criterion based feedforward equalizer (FFE) is the most common method to eliminate ISI in single-carrier system. However, there is a tradeoff between the residual ISI and enhanced noise when the system bandwidth is severely limited. Although a post filter (PF) followed by maximum likelihood sequence estimation (MLSE) can suppress the enhanced noise effectively, the residual ISI left by post-FFE will degrade the system performance as the subsequent MLSE only handles ISI introduced by PF. In this paper, an adaptive partial response equalization realized by a channel-shortening filter (CSF) is investigated, which is combined with MLSE. As the target of CSF is a partial response signal with desired low-pass feature, the noise will not be enhanced in this scheme. And the residual ISI with determined response can be fully solved by MLSE. The performance of CSF based MLSE (CSF-MLSE) is evaluated and compared to PF based MLSE (PF-MLSE) in 60/70/75Gbaud four-level pulse amplitude modulation (PAM-4) system with 10-dB bandwidth of 16.2 GHz. The experimental results show that CSF-MLSE has a better capability of ISI elimination compared to PF-MLSE. The bit error rate (BER) can be reduced from 7.4 × 10
⁻³
to 4 × 10
⁻³
and from 3 × 10
⁻²
to 1 × 10
⁻²
for 70Gbaud and 75Gbaud in optical back-to-back (OBTB) transmission. To further compensate the system nonlinear impairments, several nonlinear MLSE (NL-MLSE) techniques assisted by look-up table (LUT) or Volterra filter are proposed and investigated in this paper. The BER of 75Gbuad PAM-4 can be below 3 × 10
⁻³
after using NL-MLSE schemes.