Feature Selection with Limited Training Samples

A criterion which measures the quality of the estimates of the parameters of multivariate normal distributions for two class problems when limited number of samples are available is developed. This criterion predicts if the Hughes phenomenon occurs. The maximum number of features which does not degrade the accuracy of the classifier is then predicted. Experimental results regarding the Hughes phenomenon are included. Also presented is an example where the maximum number of features at each node in the binary tree classifier is predicted and compared with the maximum likelihood classifier. Index Terms-Training samples, multivariate normal distribution, Hughes phenomenon, feature selection, maximum likelihood classifier, bianry tree classifier.