Enhanced Three-Core Three-Mode Optical Transmission System Based on Probabilistic Shaping With Low Complexity MIMO Equalization Algorithm

A novel few-mode multi-core (FMMC) optical fiber transmission system based on probabilistic shaping (PS) with low complexity multiple-input multiple-output (MIMO) equalization method is proposed and demonstrated in this paper. Compared with the traditional FMMC counterpart, the total bit error rate (BER) is reduced by 3 times after 50km fiber transmission with the optical signal-to-noise ratio (OSNR) in the range of 12 to 20 dB. At a bit error rate of forward error correction (FEC) limit, the optical transmission system proposed in this paper outperforms the traditional one by at least 3dB OSNR improvement. When the OSNR is 20 dB and the BER is lower than the FEC limit, the transmission distance is doubled.


I. INTRODUCTION
The demand for transmission capacity has been increasing exponentially over the last decade, mainly due to the evergrowing Internet traffic. The standard single mode fiber (SSMF), currently used for long-haul transmission systems, is reaching its capacity limit and therefore new technologies are needed to be given full play to confront the challenges of higher capacity transmission [1]. Varieties of novel multiplexing schemes including space division multiplexing (SDM) have been demonstrated in the transmission experiments. For example, a dense space division multiplexed transmission over 12 cores × 3 modes few-FMMC fiber has been proposed in, and 305Tb/s transmission capacity employing 19-SDM, 100 wavelength division multiplexing polarized QPSK signals has been demonstrated in [2]- [3]. Moreover, John van Weerdenburg proposed a 138-Tb/s mode-and The associate editor coordinating the review of this manuscript and approving it for publication was Tianhua Xu . wavelength-multiplexed transmission over six-mode gradedindex fiber [4], and T. Sakamoto et al. clarify the 6-mode 7-core structure is beneficial for reducing the splice induced differential modal loss, and report the repeated 100 km transmission using 6-mode MCF and EDFA [5]. Recently, more and more researches are focused on the FMMC optical transmission system [6], in which the number of spatial channels per fiber has been further increased to achieve higher capacity. One of the main challenges in realizing FMMC optical transmission system is the interference among the modes and cores. The inevitable coupling and differential mode delay require the implementation of multiple-input multiple-output (MIMO) algorithm to equalize the signals at the receiver. The complexity and memory length of the algorithm increase with the coupling of cores/modes and delay difference between the fastest and slowest mode of the fiber.
Therefore, a MIMO equalization method based on matrix transformation (MT) is adopted to equalize the mode dispersion and core coupling with lower complexity [7].
The algorithms commonly used in the MIMO equalization include the constant modulus algorithm (CMA) [8] and decision directed Least Mean Square Algorithm (DD-LMS) [9]. But the CMA works only for the signal with a constant modulus modulation format such as 4 QAM. Convergence performance of the CMA is closely related to the selection of step size. And the CMA only considers the statistical characteristics of signal modulus value, making phase estimation a must at the receiver end. As for the DDLMS algorithm, the steady-state error of the algorithm is small, but the equalizing performance can only be exhibited after the eye diagram is basically opened and BER drops to a certain magnitude. Generally, for higher order modulation formats, such as 16 QAM, the CMA algorithm is first used to achieve pre-convergence, when the error rate is determined to fall to a certain threshold, the DDLMS algorithm is then put into work [10]. The application of the FMMC fiber in the access network has become more mature, but the performance of the BER in the relatively long distance transmission environment using the 16 QAM modulation with MIMO equalization method is not very satisfactory.
As a typical modulation format optimization technology with extensive researcher investigation, PS technology has received more and more attention due to its advantages of higher transmission capacity and lower system complexity, proven to be a promising new technology in the future [11]- [15]. Compared with conventional signaling schemes, in which signal points are uniformly distributed in the constellation, probabilistic shaping adjusts the distribution of various constellation signal points by means of intentionally and dynamically reducing the transmitting probabilities of the outer constellation signal points with larger energy, while in turn increasing the chances of the inner signal points being transmitted. Such a method can reduce the average transmitted signal power and enlarge channel capacity to a large degree, thus providing an improved BER performance. It's worth mentioning that the ultimate theoretical gain of 1.53 dB can be expected in Gaussian channel in a Maxwell-Boltzmann distribution [16].
In this paper, we propose a novel FMMC optical transmission system based on PS with low complexity MIMO equalization method. By employing this system, the performance of the BER in the relatively long distance transmission environment is satisfactory. In the simulation, we use the PS-16QAM mapping scheme and combine the MIMO algorithm to achieve channel equalization. Theoretical analysis is given to verify the favorable performance of the proposed system, along with the persuasive simulation results.

A. CHANNEL MODEL OF FEW-MODE MULTI-CORE FIBER
The few-mode multi-core (FMMC) fiber shown in Fig. 1 has three cores, and each core has three orthogonal modes (LP 01 , LP 11a and LP 11b ), which are used as independent channels to carry signals for information transmission, thereby tripling the transmission capacity of the system. In a mode multiplexing system based on multi-mode fiber, different modes can carry signals with a new degree of freedom, which can greatly improve spectrum utilization. In FMMC fibers, differential mode delay and crosstalk caused by mode or core coupling are the two main causes of signal deterioration [17]. For the transmission link with few-mode multi-core, due to the complexity of its internal crosstalk factors, the fiber segmentation model is theoretically analyzed. We assume ideal propagation along each segment m (without mode coupling) described by the diagonal matrix M (ω). The ideal propagation matrix can be expressed as [17]: We assume that the effect of random mode coupling in any fiber span can be equivalent to the effect of coupling concentrated at the end of the span [18]. For three-mode systems, the matrix K m describing the coupling between segment m and m + 1 as follows [17]: (2), as shown at the bottom of the next page, where b and ϕ 0 represent the radial offset and rotation angle, respectively. ξ is the mode field size of the LP 01 mode. The transmission matrix for each core is then given by: The cores of fiber are symmetrically distributed, the core parameters are the same, and the mode field distribution is uniform. Each core is coupled between the core at the end of the paragraph and the beginning of the next section. The coupling matrix D m for three-core system is shown as the follows: where D pq,m is the coupling coefficient for section m and the coupling coefficient can be expressed as [19]: For a three-mode three-core system, each mode is modulated and loaded with signals, denoted as U = [u 1 , u 2 , · · · , u 9 ] T . The signals of each core after the transmission matrix T (ω) is: after sampling, three signals of each core become one signal and then get a new signal matrix [Z 1 , Z 2 , Z 3 ] T . Finally, the final output signal is obtained after the core coupling matrix: The signal crosstalk exists in both modes and cores, where the crosstalk among modes is more serious than that among cores. In order to eliminate the crosstalk, MIMO equalization method is needed at the receiver, which can estimate the channel response matrix for compensation.

B. PROBABILISTIC SHAPING
As shown in Fig. 2, the randomly generated uniform binary bit sequence enters the FMMC channel after being mapped in the constellation by PS-16QAM scheme. At this time, due to the influence of factors such as mode dispersion, coupling, and crosstalk between the cores, the mapped data deteriorates seriously. After the MIMO equalization, the signal carrying severely distorted data is de-mapped by PS-16QAM to obtain an equalized binary bit sequence, which is then compared with the initial binary bit sequence for the BER calculation. PS increases the probability of these 4 constellation points in the inner circle of 16QAM, while reduces the probability of those 4 constellation points in the outer circle. Therefore, the average power of the 16QAM signal after PS will be lower than that of the original 16QAM signal. The constellation obtained by the transmitter is shown in Fig. 3(a), and the probability distribution is illustrated in Fig. 3(b). In order to make equal the average signal power in these two constellation, the constellation of the PS-16QAM needs to be expanded. The enlargement of the constellation diagram means that the Euclidean distance between the constellation points is increased and the fault tolerance is enhanced. Under the same signal-to-noise ratio (SNR), the bit error rate will decrease due to the increase of the Euclidean distance.
In probabilistic shaping, the Maxwell-Boltzmann distribution is employed, which can be expressed as [20]: where υ is the scaling factor, representing the degree of probabilistic shaping, taking values between 0 and 1. The information entropy of signals is: where the unit of information entropy is bits/symbol. In the case of the same bandwidth, the bit rate expression calculated from the information entropy is: where the scalars B baud represent the baud rate. For uniform 16QAM and PS-16QAM modulation formats, their information entropy is different. When comparing bit error rate performance, we keep the same net data rate by changing the baud rate.

C. LOW COMPLEXITY MIMO EQUALIZATION ALGORITHM
The algorithms commonly used in the MIMO equalization include the CMA and DDLMS. In this paper, we adopt a switching strategy taking both CMA algorithm and DDLMS algorithm into account. The CMA algorithm is used to reduce the BER. When the BER is reduced to a certain level, we switch to the DDLMS algorithm after the eye diagram is basically opened. Define the error function of the CMA-DDLMS algorithm as [8]: where α is a constant, used to balance the step size of the DDLMS algorithm and the CMA algorithm. e DDLMS n and e CMA n are the error functions of the DDLMS algorithm and the CMA algorithm, and their expressions are [8], [21]: In the article, for the 16QAM signals, the decision method we used is called threshold decision [22], so thatŷ (n) is in the symbol set of the transmitted signal.
If µ 1 and µ 2 are the optimization step sizes of the DDLMS algorithm and the CMA algorithm, respectively, then α is [8]: Define In Eq. (6), the value of the function f is determined by the ratio of the error function value of the DDLMS algorithm to the error function value of the CMA algorithm. Figure 4(a) shows the system block diagram of the algorithm. In the case of low signal-to-noise ratio, CMA algorithm is used to reduce the symbol error rate until the DDLMS algorithm is acceptable. If there is a sudden change in the channel, the performance of the DDLMS algorithm will be greatly reduced. At this time, the algorithm switches to the CMA algorithm again. Figure 4(b) shows a transversal filter of length L. The relationship between input u (n) and output y (n) is: where w (n) represents the tap coefficients at time n, u (n) and y (n) represent the input and output vectors at time n, then the expression of u (n) and w (n) are: w (n) = [w 0 (n) , w 1 (n) , w 2 (n) , · · · , w L−1 (n)] T , updating the tap coefficient at the next moment according to the error value e n . The expression of the iterative process can VOLUME 8, 2020 When the tap coefficient is updated at n+1, the new sequence u (n) is sent, and iteratively continues to complete the update of the output sequence y (n). For FMMC system with N spatial modes and C cores, the MIMO equalizer needs (N × C) 2 FIR filters [23]. In this paper, we adopt a MIMO DSP equalizer based on MT that can reduce the number of filters to (N 2 × C + C 2 ) that can realize MIMO DSP process with smaller scale. As shown in formula (11), a tap matrix with (N × C) 2 dimensions is transformed into C small tap module matrices A C N ×N and a core matrix B 1 C×C . Where w ij and k ij represents an FIR filter of length L. The main function of matrix A is to balance the damage caused by multiplexing coupling between various modes in the same fiber core, and the function of matrix B is to balance the damage caused by the interaction between the cores.
Based on the above theoretical analysis, we propose a MIMO equalizer structure suitable for a three-mode threecore optical fiber transmission system as shown in Fig. 5. The input signals on the left side of the equalizer are equalized by chromatic dispersion, and then enter the mode matrix to eliminate the cross-mode interaction. The signal is resampled and then sent to the core matrix to eliminate inter-core crosstalk. The transmission and reception of a MIMO system can be expressed as: where X is the transmitted signal, U is the sampling matrix of the received signal, S is the sampling matrix of the noise and H is the channel matrix.
Without considering noise, the output signal of the MIMO equalizer is an estimation of the transmitted signal sequence  x j by the equalizer, namely: where W is the tap matrix and can be expressed by formula (11). It can be seen from the above formula that the tap matrix and the channel matrix can be approximately regarded as mutually inverse matrices. In general, as an algorithm specialized in back-end signal processing, MIMO equalization does not affect the mapping scheme of front-end probability shaping. The two techniques are cascaded together to improve the overall performance of the system. VOLUME 8, 2020

III. SIMULATION AND RESULTS
The setup of the system is shown in Fig. 6, where we adopt three modes and three cores to demonstrate the proposed method. The key simulation parameters are given in TABLE 1. Each core supports three lowest-order linearlypolarized modes (LP 01 , LP 11a and LP 11b ), which are predicted to have the lowest propagation losses. At the transmitter, a total of nine PS-16QAM modulation sequences are generated and divided into three streams, which are respectively sent to the three cores. Three QAM signals are combined and coupled into one core via a mode multiplexer (Mux). After mode multiplexing, these three optical signals are injected into the three cores of FMMC fiber through the fan-in device. In our simulation, the FMMC fiber stretches from 50 km to 130 km due to the fact that it was carried out in a relatively long distance transmission environment. After FMMC fiber transmission, the signals are fed into three mode de-multiplexers (DeMux) through fan-out device. The respective outputs from mode DeMux are guided into the multi-mode coherent receiver detection. After analog to digital conversion, the nine digital symbols streams are equalized by our proposed MIMO method. Finally, after demapped by  PS-16QAM, 9 binary bit streams are obtained and compared with the initial number of binary bits to calculate the bit error rate to evaluate the performance of the transmission system.
As shown in Fig. 7, a, b, and c represent received constellations of three modes of the core 1, d, e, and f represent received constellations of the three modes of the core 2, and g, h, and i represent received constellations of three modes of the core3 respectively. After a transmission link based on a FMMC, the constellation diagrams of the LP 01 mode, LP 11a mode, and LP 11b mode signals of the three cores before equalization have been blurred due to the effects of intermode and intercore effects. After the proposed MIMO equalization method, the Uniform 16QAM constellation diagram of the three modes of each core is shown in Fig. 8. Fig. 9 shows the PS-16QAM constellation diagram of the three modes of each core after the proposed MIMO equalization method. Comparing these two methods, it is obvious that the discrimination of the constellation points in the PS-16QAM  constellation diagram after the proposed MIMO algorithm is significantly improved. Figure 10 and 11 illustrate the BER performance of signal on the three cores and modes respectively. It can be seen that the performance improves a lot after proposed equalization for both cores and modes. Figure 10 shows the averaged BER performance of the three modes for each core, where the BER performances are almost the same for the three cores. Under the MIMO equalization algorithm using the PS-16QAM mapping scheme, all the three cores achieve BER below FEC limit (BER= 3.8 × 10 −3 ) when the OSNR is above 16 dB. Due to the similar performance of the three cores, we have chosen one core to investigate the performance of signal on the three modes, which is plotted in Fig. 11. After the same equalization algorithm, all the three modes have got BER performance under FEC limit with OSNR of 17 dB. Obviously, when the mapping scheme of PS-16QAM is adopted, the noise tolerance of the transmission system is significantly improved.
As the transmission distance increases, the transmission performance of the FMMC system will continue to decrease. When the transmission distance increases to 50 km, the BER has exceeded the forward FEC limit under the traditional MIMO equalization method for both cores and modes, as shown in Figs. 12 and 13. Figure 12 shows the averaged BER performance of the three modes for each core, where the BER performances are almost the same for the three cores. When we adopt PS-16QAM encoding scheme, all the three cores achieve BER under FEC limit until the transmission distance is above 90 km. Analogously, owing to the similar performance of the three cores, we have chosen one core to investigate the performance of signal on the three modes, which is plotted in Fig. 13. When the transmission distance is less than 85 km, all the three modes have got BER performance under FEC limit. In general, when the SNR is unchanged, the PS-16QAM mapping scheme is used, and the transmission distance of the system is greatly improved.

IV. CONCLUSION
We have proposed and demonstrated an enhanced three-mode three-core optical transmission system based on probabilistic shaping technology with low complexity MIMO equalization algorithm. The adoption of a mapping scheme of PS-16QAM based on MIMO algorithm, can effectively reduce the BER in the long-distance transmission environment. When the transmission distance is 50 km, the OSNR is higher than 17 dB and the total bit error rate is lower than the FEC limit. When OSNR is 20 dB, the transmission distance is no more than 85 km and the total bit error rate does not exceed the FEC limit. The results indicate that the proposed FMMC method is a promising candidate for the long-haul transmission system. SONGSONG QU received the bachelor's degree in integrated circuit design and systems integration from Nantong University, China. He is currently pursuing the master's degree with the School of Physics and Optoelectronics, Nanjing University of Information Science and Technology (NUIST), China. His research interests include optical communication, passive optical networks, and optical signal processing.