A low-complexity generalized sphere decoding approach for underdetermined linear communication systems: performance and complexity evaluation

For underdetermined linear systems, original sphere decoding (SD) algorithms fail due to zero diagonal elements in the upper-triangular matrix of the QR or Cholesky factorization of the underdetermined matrix. To solve this problem, this paper presents a low-complexity generalized sphere decoding (GSD) approach by transforming the original underdetermined problem into the full-column-rank one so that standard SD can be directly applied on the transformed problem. Since the introduced transformation maintains the dimension of the original problem for all M-QAM's, the proposed GSD approach provides significant reduction in complexity as compared to other GSD schemes, especially for M-QAM with large signaling constellation. Both performance and expected complexity are analyzed to provide the comprehensive relationships between the performance and complexity of the proposed GSD and its parameters. Illustrative simulation and analytical results are in good agreement in terms of both the performance and complexity and indicate that with the properly selected design parameters, the proposed GSD scheme can approach the optimum maximumlikelihood decoding (MLD) performance with low complexity for underdetermined linear communication systems including underdetermined MIMO systems, and the proposed expected complexity analysis can be used as reliable complexity estimation for practical implementation of the proposed algorithm and serve as reference for other GSD algorithms.