Random Matrix Theory Predictions of Dominant Mode Rejection Beamformer Performance

Adaptive beamformers use a sensor covariance matrix estimated from data snapshots to mitigate directional interference and attenuate uncorrelated noise. Dominant mode rejection (DMR) is a variant of the classic minimum variance distortionless response (MVDR) algorithm that replaces the smallest eigenvalues in the covariance estimate by their average while leaving the largest ones unmodified. Since DMR does not invert the small eigenvalues of the sample covariance, it achieves a higher white noise gain than MVDR while still suppressing loud interferers, thereby yielding a higher signal-to-interference-plus-noise ratio (SINR). MVDR achieves an average output SINR within 3 dB of the SINR achievable with the true covariance with approximately twice as many snapshots as sensors. Prior empirical analyses showed that achieving the same SINR with DMR only requires approximately twice as many snapshots as interferers. This paper derives analytical models of white noise gain and interference leakage for DMR using random matrix theory (RMT), specifically the eigenvalue and eigenvector limiting results of the spiked covariance model. Applying the new analytical models confirms the empirical DMR snapshot requirement. To handle finite cases with a large number of interferers relative to the array size, this paper derives a modified spiked covariance model that improves the accuracy of RMT results, and hence DMR performance predictions.