On Kleinberg's Stochastic Discrimination Procedure

A new condition for high accuracy on test sets is given for the method of stochastic discrimination (SD), a pattern recognition method introduced by Kleinberg. This condition provides a simple explanation for the observed good generalization properties and overtraining-resistance. We also show that the method of SD remains valid if the original assumption of uniform distribution on a finite space for resampling is relaxed.