Performance of maximum likelihood sequence estimation with different modulation formats

The performance of several common modulation formats in conjunction with MLSE in systems dominated by thermal noise in the receiver is evaluated. The results indicate that the dispersion limited reach can be substantially increased by the application of MLSE. The performance of the duobinary format in conjunction with MLSE is still better then NRZ and RZ under the same conditions. For the amounts of accumulated dispersion considered, the gain obtained by using MLSE including low resolution A/D conversion in conjunction with duobinary modulation is 2.0 dB with respect to NRZ at the expense of a more complicated transmitter.