Performance of the Low-Rank Adaptive Normalized Matched Filter Test Under a Large Dimension Regime

When a possible target is embedded in a low-rank (LR) Gaussian clutter (which is contained in a low-dimensional subspace) plus a white Gaussian noise, the detection process can be performed by applying the LR adaptive normalized matched filter (LR-ANMF), which is a function of the estimated projector. In a recent work, we derived an approximate distribution of the LR-ANMF under the ℌ0 hypothesis by using a restrictive hypothesis (the target has to be orthogonal to the clutter subspace). In this paper, we propose to determine new approximations of the Pfa and the Pd of the LR-ANMF by relaxing this restrictive hypothesis. This new derivation is based on results concerning the convergence in a large dimension regime of quadratic forms. Simulations validate our result, in particular, when the tested signal is close to the clutter subspace.