A case study on bagging, boosting and basic ensembles of neural networks for OCR

We study the effectiveness of three neural network ensembles in improving OCR performance: basic, bagging, and boosting. Three random character degradation models are introduced for training individual networks in order to reduce error correlation between individual network and to improve the generalization ability of neural networks. We compare the recognition accuracies of these three ensembles at various rejection rates. It is shown that although the boosting ensemble is slightly more accurate than the basic and bagging ensembles at zero rejection rate, the advantage of the boosting training over the basic and bagging ensembles quickly disappears as more patterns are rejected. Eventually the basic and bagging ensembles outperform the boosting ensemble at high rejection rates. Explanation of such a phenomenon is provided. We also apply the optimal linear combiner to each of the three ensembles to capture different error correlation characteristics of the three ensembles.