Mutual information and intrinsic dimensionality for feature selection

In this article we proposed a feature selection method based on mutual information (MI) and intrinsic dimensionality (ID) estimators. First, MI ranks the normalized feature space in accordance to minimal-redundancy-maximal-relevance (mRMR) criterion. Next, ID estimates the minimum number of features to represent the observed properties of the data. Two techniques of ID were tested: principal component analysis (PCA) and maximum likelihood estimator (MLE). Support vector machine (SVM) was used to classify five medical datasets. Receiver operating characteristics (ROC) analysis evaluated the classification performance before and after feature selection. Results showed that MI and ID are effective techniques for feature selection to reduce the classification error.