Supervised Linear Dimensionality Reduction Based on Factor Analysis and Information-Theoertic Criteria

Recently supervised linear dimensionality reduction method based on Information-Theoretic Criteria has attracted more attention. In existing methods, the first and second order statistics of the data in the original high-dimensional space is estimated, and then the transformation matrix based on the information-theoretic criteria is designed. However, it is difficult to accurately estimate these statistics in the original high-dimensional space, especially in the case of limited data. Thus the transformation matrix obtained may be nonoptimal, which will affect the final classification performance. To solve this problem, a novel supervised linear dimensionality reduction method is proposed in this paper. In our method, the statistical structure of the transformed low-dimensional subspace is described via factor analysis (FA) model, while the mutual information (MI) between the transformed data and their class labels is maximized. Moreover, the joint optimization of MI function and log-likelihood function is achieved, which can not only reduce the estimation errors, but also ensure the separability of the transformed data. Experimental results on benchmark datasets and radar data demonstrate the effectiveness of our method.