Performance Evaluation of Automatic Feature Discovery Focused within Error Clusters

We report performance evaluation of our automatic feature discovery method on the publicly available Gisette dataset: a set of 29 features discovered by our method ranks 129 among all 411 current entries on the validation set. Our approach is a greedy forward selection algorithm guided by error clusters. The algorithm finds error clusters in the current feature space, then projects one tight cluster into the null space of the feature mapping, where a new feature that helps to classify these errors can be discovered. This method assumes a ``data-rich'' problem domain and works well when large amount of labeled data is available. The result on the Gisette dataset shows that our method is competitive to many of the current feature selection algorithms. We also provide analytical results showing that our method is guaranteed to lower the error rate on Gaussian distributions and that our approach may outperform the standard Linear Discriminant Analysis (LDA) method in some cases.