Inconsistency of ESPRIT DoA Estimation for Large Arrays and a Correction via RMT

Recent advances in random matrix theory (RMT) have shed light on the limitations of the sample covariance matrix (SCM) as an estimator for population covariance in large-dimensional scenarios. The impact of these findings on the widely used SCM-based ESPRIT Direction-of-Arrival (DoA) estimation technique, however, remains unclear. This paper addresses this issue by examining the asymptotic behavior of ESPRIT in the context of large arrays with limited samples, where the number of samples/snapshots $T$ and the number of sensors $N$ are both large and comparable. We demonstrate that classical ESPRIT produces inconsistent DoA estimates as N, T → ∞ at the same pace. To overcome this, we propose the G-ESPRIT method and prove its asymptotic consistency in the same setting. Numerical simulations are provided to validate our theoretical results.