Efficient Beamspace-Based Algorithm for Two-Dimensional DOA Estimation of Incoherently Distributed Sources in Massive MIMO Systems

In massive multiple-input multiple-output (MIMO) systems, accurate direction-of-arrival (DOA) estimation is critical for the transmitters to conduct downlink precoding. In this paper, we focus on the problem of two-dimensional DOA estimation of incoherently distributed sources in massive MIMO systems using uniform rectangular arrays (URAs). First, with the generalized array manifold, we obtain the beamspace array manifold by performing beamspace transformation on the observed vector of the URA, and establish one beamspace shift invariance structure via appropriate beamforming matrix design and array manifold selection. Second, from the beamspace array manifold, we extract two array manifolds, and derive the other beamspace shift invariance structure. Final, the total least squares approach is used to estimate the nominal azimuth and elevation DOAs. With the DOA estimates, the corresponding angular spreads are given in closed-form solutions. Our approach avoids spectral search and reduces the dimensions in matrix operations, thus, it is more computationally attractive than the existing techniques. Numerical results show that the proposed algorithm achieves performance comparable to that of the conventional estimators in massive MIMO systems.