1 Introduction
Boosting has attracted a lot of research interests since the first practical boosting algorithm, AdaBoost, was introduced by Freund and Schapire [1]. The machine learning community has spent much effort on understanding how the algorithm works [2], [3], [4]. However, to date there are still questions about the success of boosting that are left unanswered [5]. In boosting, one is given a set of training examples , , with binary labels being either or . A boosting algorithm finds a convex linear combination of weak classifiers (a.k.a. base learners, weak hypotheses) that can achieve much better classification accuracy than an individual base classifier. To do so, there are two unknown variables to be optimized. The first one is the base classifier. An oracle is needed to produce base classifiers. The second one is the positive weights associated with each base classifier.