Target detection in clutter background: Null or whiten the clutter

In a clutter-dominant target detection problem, an adaptive detector needs to either whiten or null the clutter prior to signal detection. To whiten clutter, it is required to estimate the covariance matrix of a received clutter, while a proper set or estimate of the parameters of clutter subspace model is required to null the received clutter. In this paper, we investigate the detection performance of a subspace-based detector and a sample covariance matrix(SCM)-based detector presented in [1] and [2], respectively, but for a mono-static radar system. Our simulation results show that the subspace-based detector not only can reach a predetermined probability of false alarm (fully nulling clutter) but also offers better detection performance as compared with its counterpart. In this case, the SCM-based detector only offers better detection performance for detecting targets with Doppler frequency resided in clutter region as compared with that of subspace-based one.