Second-Order Perturbation-Based Nonlinearity Compensation Using BiLSTM Neural Network in Optical Transmission Systems

In coherent optical transmission systems, fiber nonlinearity has become a major barrier against performance improvement. The first-order (FO) perturbation-based nonlinearity compensation (PB-NLC) has been extensively studied and combined with machine learning techniques. However, its performance is constrained in long-haul transmission systems, as higher-order interference becomes more significant. In this work, we investigate a second-order (SO) perturbation-based nonlinearity compensation scheme using a bidirectional long short-term memory (BiLSTM) neural network. Simulation results show that the proposed SO perturbation network provides an improved Q² value of 1.86 dB compared with the dispersion compensation scheme after 1200 km transmission and reduces the computational complexity by 86.09%, compared to the FO-PB-NLC using a fully-connected neural network. In a 6-channel transmission system, the proposed SO-BiLSTM scheme improves the Q² value by 1.19 dB while reducing the computational complexity by 84.12%.