An investigation on digital signal processing for coherent optical communication system with advanced modulation format

In recent years, many new applications such as cloud computing, Internet of Things (IoT), artificial intelligence (AI) are developing rapidly, which put large capacity requirements on communication network. Coherent optical communication systems based on advanced modulation format have been extensively studied to improve transmission capacity. High-order modulation format requires a better signal-to-noise ratio (SNR) for signal receiving. However, the various damages in transmission links would decrease SNR. In this paper, digital signal processing (DSP) techniques are investigated to compensate signal damages, including Time-domain equalization (TDE) algorithm to compensate chromatic dispersion (CD), constant mode algorithm (CMA) and direct decision least mean square (DD-LMS) algorithm to compensate polarization mode dispersion (PMD), fourth power algorithm to correct frequency deviation, Viterbi & Viterbi and Blind Phase Search (BPS) algorithms to estimate phase noise. Moreover, constellation shaping technology is also discussed to make the transmission capacity closer to Shannon limit.