Optimal Design of Dispersion Filter for Time-Domain Split-Step Simulation of Pulse Propagation in Optical Fiber

The nonlinear Schrödinger equation can be solved by split-step methods, where in each step, linear dispersion and nonlinear effects are treated separately. This paper considers the optimal design of an FIR filter as the time-domain implementation for the linear part. The objective is to minimize the integral of the squared error between the FIR frequency response and the desired dispersion characteristics over the band of interest. This least square (LS) problem is solved in two approaches: the normal equation approach gives the explicit solution, whereas the singular value decomposition approach, which is based on the theory of discrete prolate spheroidal sequences, provides geometrical insights and reveals that the normal equation could be ill-conditioned. In addition, the frequency response might exhibit singular behaviors such as overshoot. We propose two filters that both can mitigate these shortcomings: the regularized LS filter achieves this by adding a regularization term to the objective function; the quadratically constrained quadratic programming-based filter addresses overshooting more efficiently by imposing a maximum magnitude constraint on the frequency response. Numerical results show that these filters can suppress the overshoots, control the squared error, reduce the filter length and lower the computational complexity. For both single channel and wavelength-division multiplexing channels, the proposed methods generate similar outputs as the standard split-step Fourier method.