A selective incremental approach for transductive nearest neighbor classification

Transductive learning is a special case of semi-supervised learning, where class labels to the test patterns alone are found. That is, the domain of the learner is the test set alone. Often, transductive learners achieve a better classification accuracy, since additional information in the form of test patterns location in the feature-space is used. For several inductive learners, there exists corresponding transductive learners; like for SVMs there exists transductive SVMs (TSVMs). For nearest neighbor based classifiers, their corresponding transductive methods are achieved through graph mincuts or spectral graph mincuts. It is shown that these solutions achieve low leave-one-out cross-validation (LOOCV) error with respect to nearest neighbor based classifiers. It is formally shown in the paper that, through a clustering method, it is possible to get various solutions having zero LOOCV error with respect to nearest neighbor based classifiers. Some solutions can have low classification accuracy. The paper proposes, instead of optimizing LOOCV error, to optimize a margin like criterion. This criterion is based on the observation that similar labeled patterns should be nearer to each other, while dissimilar labeled patterns should be far away. An approximate method to solve the proposed optimization problem is given in the paper which is called selective incremental transductive nearest neighbor classifier (SI-TNNC). SI-TNNC finds the test pattern from the test set which is very close to one class of training patterns and at the same time very much away from the other class of training examples. The selected test pattern is given its nearest neighbor's label and is added to the training set. This pattern is removed from the test set. The process is repeated with the next best test pattern, and is stopped only when the test set becomes empty. An algorithm to implement SI-TNNC method is given in the paper which has a quadratic time complexity. Other related solutions have either cubic time complexity or are NP-hard. Experimentally, using several standard data-sets, it is shown that the proposed transductive learner achieves on-par or better classification accuracy than its related competitors.