Power spectral analysis of spectral shaping trellis coded modulation

A nonlinear binary code with a single carrier M-QAM linear modulation is designed to control certain transitions in the symbol constellation. This concept is employed to develop spectral shaping modulation codes with M-QAM linear modulations so as to reduce out-of-band power. The code is realized as a nonlinear binary convolutional code, and the coded modulation is represented as a Markov chain. The closed form of the power spectral density for this trellis coded modulation is derived.