On two measures of classifier competence for dynamic ensemble selection - experimental comparative analysis

This paper presents two methods for calculating competence of a classifier in the feature space. The idea of the first method is based on relating the response of the classifier with the response obtained by a random guessing. The measure of competence reflects this relation and rates the classifier with respect to the random guessing in a continuous manner. In the second method, first a probabilistic reference classifier (PRC) is constructed which, on average, acts like the classifier evaluated. Next the competence of the classifier evaluated is calculated as the probability of correct classification of the respective PRC. Two multiclassifier systems (MCS) were developed using proposed measures of competence in a dynamic fashion. The performance of proposed MCS's were compared against six multiple classifier systems using six databases taken from the UCI Machine Learning Repository and Ludmila Kuncheva Collection. The experimental results clearly show the effectiveness of the proposed dynamic selection methods regardless of the ensamble type used (homogeneous or heterogeneous).