Unsupervised Feature Selection: Minimize Information Redundancy of Features

This paper proposes an unsupervised feature selection method to remove the redundant features from datasets. The major contributions are twofold. First, we propose an eigen-decomposition method to rank the hyperplanes (which describes the relations between features) based on their linear dependency characteristic, and then design an efficient Gaussian-elimination method to sequentially remove the feature that is best represented by the rest of the features. Second, we provide a proof showing that our method is similar to removing the features that contribute the most to the Principal Components with the smallest eigenvalue, but considering the effect of each removal of features with complexity about max(O(nm), O(n
²
)) instead of O(n
³
), where n is the number of features and m is the number of observations. We perform experiments on an artificial and real-world datasets. The results show that our method can almost perfectly remove those dependent features without losing any independent dimension in the artificial dataset and outperforms two other competitive algorithms in the realworld datasets.