Radar Clutter Covariance Estimation: A Nonlinear Spectral Shrinkage Approach

In this article, we exploit the spiked covariance structure of the clutter plus noise covariance matrix for radar signal processing. Using state-of-the-art techniques high dimensional statistics, we propose a nonlinear shrinkage-based rotation invariant spiked covariance matrix estimator. We state the convergence of the estimated spiked eigenvalues. We use a dataset generated from the high-fidelity, site-specific physics-based radar simulation software RFView to compare the proposed algorithm against the existing rank constrained maximum likelihood (RCML)-expected likelihood (EL) covariance estimation algorithm. We demonstrate that the computation time for the estimation by the proposed algorithm is less than the RCML-EL algorithm with identical Signal to Clutter plus Noise (SCNR) performance. We show that the proposed algorithm and the RCML-EL-based algorithm share the same optimization problem in high dimensions. We use Low-Rank Adaptive Normalized Matched Filter (LR-ANMF) detector to compute the detection probabilities for different false alarm probabilities over a range of target signal-to-noise ratios (SNR). We present preliminary results which demonstrate the robustness of the detector against contaminating clutter discretes using the challenge dataset from RFView. Finally, we empirically show that the minimum variance distortionless beamformer error variance for the proposed algorithm is identical to the error variance resulting from the true covariance matrix.