PGFB: A hybrid feature selection method based on mutual information

Feature selection is a crucial step in the supervised learning process. Traditional feature selection methods based on mutual information cannot directly handle the feature set with hybrid continuous and categorical features, and cannot dynamically eliminate the redundant features in the feature selection process. Resort to mutual information, a hybrid feature selection method named PGFB is proposed in this paper. Parzen window based General mutual information estimation method (PG) is proposed in this paper to handle the hybrid input feature set and the regression problem in a direct way. As an improvement to the Sequential Forward Floating Search (SFFS) method, a Forward/Backward sequential search method (FB) without predefining the number of selected features is proposed to eliminate redundant features dynamically so as to obtain a more effective feature subset. Numerical tests are thoroughly carried out to compare the proposed PGFB method with other seven feature selection methods based on mutual information. Six data sets and five types of classifiers are adopted for testing classification performance. One data set is adopted for testing regression performance. A case study on real-life power system is introduced briefly. Numerical results show the effectiveness of the proposed method.