Bayesian Robust Signal Subspace Estimation in Non-Gaussian Environment

In this paper, we focus on the problem of low rank signal subspace estimation. Specifically, we derive new subspace estimator using the Bayesian minimum mean square distance formulation. This approach is useful to overcome the issues of low sample support and/or low signal to noise ratio. In order to be robust to various signal distributions, the proposed Bayesian estimator is derived for a model of sources plus outliers, following both a compound Gaussian distribution. In addition, the commonly assumed complex invariant Bingham distribution is used as prior for the subspace basis. Finally, the interest of the proposed approach is illustrated by numerical simulations and with a real data set for a space time adaptive processing (STAP) application.