Optimizing Complete Cross-Validation for Prototype Weighting in Nearest Neighbor Classification

The Nearest Neighbor (NN) classifier is one of the most used and well-known techniques for classification. However, NN suffers from several drawbacks such as high storage requirements, low efficiency in classification response, and noise sensitivity in the neighborhood. Prototype weighting as one the effective learning methods, assigns a weight to each prototype in order to determine its importance and simultaneously removes both redundant and noisy ones as a side effect. NN has an instance-based structure, and this is why; it is one of the rare methods for which the results of cross validation can be achieved for all possible combinations called as Complete-Cross-Validation (CCV) without need to construct and train exponential number of models. Using CCV as the objective function in learning the weights of prototypes can increase generalization capability of the classifier and also decrease the randomness of the learning method. Nevertheless, it cannot be computed such efficiently to be used as a fitness function during most of learning algorithms. In this paper, an iterative learning method has been proposed, in which the objective is incrementally improved without computing CCV even once. The proposed approach was assessed through a series of experiments with UCI in comparison with basic NN and related prototype weighting scheme and could outperform them.