Complementary classifier design using difference principal components

In classifier combination, the degree of recognition accuracy improvement depends not only on how to combine classifiers, but also on how much classifiers complement each other. In order to improve accuracy, therefore, it is important to design complementary classifiers with respect to each other. The authors propose a method to design a classifier complementary to an existent one, which satisfies the requirements: (a) it can recognize patterns misrecognized by the existent classifier with high accuracy, and (b) the number of patterns which are correctly recognized by the existent classifier but turn out to be misrecognized after combination can be minimized. In the proposed method, features are obtained by projection of original features onto axes such that the scatter of projection of patterns of a given class is small and that the squared mean of projection of patterns misrecognized in the given class is large. As the discriminant function, Fisher's linear discriminant function is applied using not only linear terms but also quadratic terms. Through experiments using handwritten numeral data included in the NIST database, it has been confirmed that the requirements mentioned above are satisfied. The misrecognition rate reduce to 56% for training data and to 86% for test data.