Optimal soft decoding for combined trellis-coded quantization/modulation

The maximum a posteriori (MAP) algorithm is used as a minimum mean-squared-error decoder for combined trellis-coded quantization and modulation. The optimal trellis-coded quantizer is derived when the soft-decoding MAP algorithm is used as a decoder. The trellis-coded quantizer and the soft decoder are optimized iteratively for minimum overall distortion from transmitter input to receiver output. Significant performance improvement is achieved for both memoryless Gaussian and uniform source, especially for very noisy channels.