Joint transmitter-receiver optimization for partial response channels based on nonmaximally decimated filterbank precoding technique

Transmitter and receiver filter design plays a key role in communication systems with partial response channels. For a partial response system, a nonmaximally decimated multirate filter bank can be used as a precoder. Based on the MMSE criterion, a closed-form solution to the joint transmitter-receiver optimization problem for noisy partial response channels is presented in this paper. It is found that the redundancy introduced by the nonmaximally decimated filter bank precoder can compensate for spectral nulls in partial response channels that can impair several known joint optimization methods. Simulation results corroborate the analysis that joint transmitter-receiver optimization can achieve significant performance improvement over inverse filter receiver and receiver optimization schemes.