Adaptive online learning of optimal decision boundary using active sampling

An adaptive online learning method is presented to facilitate pattern classification using active sampling to identify optimal decision boundary for a stochastic oracle with a minimum number of training samples. The strategy of sampling at the current estimate of the decision boundary is shown to be optimal in the sense that the probability of convergence toward the true decision boundary at each step is maximized, offering theoretical justification on the popular strategy of category boundary sampling used by many query learning algorithms.