An improved design of transmit digital and receive analog filters to combat timing jitter

This paper addresses a novel technique for designing Nyquist filters which have potential application for efficient implementation of high-speed data transmission systems. The Nyquist response is chosen to be a result of the combination of a transmit pulse-shaping filter and a receive front-end filter. We realize that in high data-rate applications, implementation of the receiver front-end filter in digital form is usually costly, because of the high throughput of the system. On the other hand, the transmitter pulse-shaping filter can be efficiently implemented in a digital form using a look-up table. This point which has been noted in some previous publications is emphasized in this paper. We propose a new formulation of the problem which optimizes both the transmit and receive filters for achieving a joint minimization of noise, intersymbol interference (ISI), and adjacent channel interference (ACI), subject to a fixed transmit power. We also introduce a penalty function so that a Nyquist pulse shape with good robustness to timing jitter is achieved.