Combining One-vs-One Decomposition and Instance-Based Learning for Multi-Class Classification

Multi-class classification is one of the most important supervised learning problems and can be solved by either designing direct multi-class classifiers (direct strategy) or decomposing it into a set of binary classification problems (indirect strategy). Direct strategy only needs training one unified classifier while indirect strategy, especially the one-vs-one decomposition method, has shown its superiority and been utilized by some popular software packages. In this article, a first attempt towards bridging the gap between direct strategy and one-vs-one decomposition for multi-class classification is conducted, and accordingly a novel approach named CODIL is proposed. Specifically, CODIL firstly transforms the class vector into a ternary label matrix (only with {-1, 0, +1}) via one-vs-one rule, where each column of the label matrix corresponds to a pair of classes. Then, CODIL determines the binary label vector (only with {-1, +1}) for unseen instance by exploiting the manifold structure information residing in its k nearest neighbors, where each element of the label vector denotes the unseen instance's prediction for its corresponding class pair. Finally, the class for unseen instance is returned via majority voting based on the binary label vector. Extensive comparative studies are conducted between CODIL and six well-established multi-class approaches over seventeen benchmark multi-class data sets. The experimental results show the superiority of the proposed CODIL approach against the compared approaches.