Radar Clutter Covariance Estimation: A Nonlinear Spectral Shrinkage Approach

In this paper, we exploit the spiked covariance structure of the clutter plus noise covariance matrix for adaptive radar signal processing. Using state-of-the-art techniques from mathematical finance and high dimensional statistics we propose a non-linear shrinkage-based rotation invariant spiked covariance matrix estimator. We compare the proposed estimator with Rank Constrained Maximum Likelihood (RCML)-Expected Likelihood (EL) covariance estimator using the Challenge dataset generated from RFView. We demonstrate that the computation-time for the proposed estimator is less than the RCML-EL estimator with identical Signal to Clutter plus Noise (SCNR) performance for the Challenge dataset. We derive the lower bound and upper bound for the normalized SCNR and empirically show that RCML-EL and the proposed estimator perform within these derived bounds for the Challenge dataset. We state the convergence for the spiked eigenvalues of the estimator.