KalmanTune: A Kalman Filter Based Tuning Method to Make Boosted Ensembles Robust to Class-Label Noise

Boosting has been shown to be a very effective approach to training ensemble classification models. Although they perform very well, boosting algorithms are sensitive to class-label noise (where training data instances are mislabelled). As the level of class-label noise in the training dataset increases, the generalisation performance of ensembles trained using boosting decreases. This paper introduces KalmanTune, a tuning process that can be applied to ensemble models after they have been trained using a boosting algorithm that reduces the impact of class-label noise. KalmanTune frames the tuning of a trained ensemble model as a static state estimation problem that can be addressed using a Kalman filter. This approach exploits the sensor fusion capability of the Kalman filter to reduce the impact of class-label noise on the trained ensemble. This paper describes KalmanTune and an evaluation experiment performed using 34 multi-class datasets with 5 levels of synthetically induced class-label noise that demonstrates that applying KalmanTune after training can improve the performance of ensemble models trained using boosting, especially when training data contains noisy class-labels.