A joint Viterbi algorithm to separate cochannel FM signals

This paper presents a method for separating cochannel FM signals. We show that the Viterbi algorithm, traditionally limited to estimation of digital sequences, can jointly track analog FM signals by separately quantizing the derivatives of their instantaneous frequencies. We employ per-survivor processing in the trellis to estimate unknown channel effects. The approach works well when the signal to interference ratio (SIR) is less than or equal to zero, in contrast to conventional interference suppression algorithms that degrade as the SIR approaches zero and fail catastrophically when the SIR<0. Comparisons of mean squared error (MSE) between the estimates and the true signals are given for varying SIR, SNR, Doppler offsets, and frequency deviations. The same approach can also be used for any other continuous phase modulation scheme, such as continuous-phase frequency-shift keying (CPFSK).