A sphere decoding approach for the vector Viterbi algorithm

High speed multi-input multi-output (MIMO) communication systems suffer from inter-channel and inter-symbol-interference (ICI and ISI). The vector Viterbi algorithm (VVA) is a maximum likelihood sequence detection (MLSD) algorithm for MIMO frequency selective channels. MLSD algorithms are desired because they minimize the probability of sequence detection error. However, they suffer from very high computational complexity. In this work, we show how a sphere decoding like algorithm can be used to reduce the complexity of VVA while preserving its optimality. For a 2 × 2 MIMO system with 16-QAM signal constellation, our algorithm cuts VVA's complexity by 50% at an SNR of 10 dB and by 60% at an SNR of 15 dB.