Underdetermined AOA Estimation of Correlated Sources Using Hybrid Beamforming Based on NSC Structure

Inheriting the enhanced degree of freedom (DOF) from nonuniform array processing, the hybrid beamforming (HBF) structure based on interleaved nonuniform subarray connection has been shown to perform well when the number of signal sources to be estimated is larger than the number of radio frequency (RF) chains. However, the angle-of-arrival (AOA) estimation may fail if the signal sources are correlated (e.g., when there are multiple arriving paths belonging to the same signal source). In this paper, we intend to exploit the enhanced DOF of the HBF with nonuniform subarray connection (NSC) in the presence of correlated signals. We investigate the effect of correlated signals on the difference co-array and propose two-dimensional (2D) indexing for AOA estimation. By adopting the compressed sensing (CS) framework to identify the 2D indices corresponding to possible signal sources, the underde-termined AOA estimation is achieved. In addition, correlation (e.g., multipath) association can be conducted using the index information. Simulation results show that the proposed method can effectively achieve underdetermined AOA estimation when there is a mixture of correlated and uncorrelated signals. This is the first such result for the NSC- HBF system.