GMI Optimisation for End-to-End Learning-based Probabilistic Constellation Shaping in 400-Gbits/s/λ DP-64QAM Optical Communication System

Autoencoders (AEs) as End-to-end (E2E) learning approaches have demonstrated the capability to achieve robust constellation shaping in optical fibre communication systems. However, in terms of maximising the generalised mutual information (GMI), the existing AE-based probabilistic constellation shaping (PCS) schemes, which combine the binary cross-entropy (BCE) and source entropy for GMI estimation as the loss function, cannot fully deliver the shaping gain. For PCS, the reason for this is that both BCE and source entropy are affected by changes in the probability distribution of square quadrature amplitude modulation (QAM) constellation points, which will keep the learned probability distribution of constellation points near the uniform distribution. In this work, we have developed a novel AE-based PCS with a modified loss function by adding an adaptive weight factor to dynamically control the influence of BCE and source entropy on GMI. The GMI optimised by the proposed learnt PCS behaves similarly to the Maxwell-Boltzmann (MB) distribution for an additive white Gaussian noise (AWGN) channel. We also train the proposed AE-based PCS in a $400-\mathrm{Gbits} / \mathrm{s} / \lambda$ dual-polarisation 64QAM (DP64QAM) wavelength-division multiplexing (WDM) optical fibre communication system.