On-line active learning based on enhanced reliability concepts

In this paper, we present a new methodology for conducting active learning in a single-pass on-line learning context, thus reducing the annotation effort for operators by selecting the most informative samples, i.e. those ones helping incremental, evolving classifiers most to improve their own predictive performance. Our approach will be based on certainty-based sample selection in connection with version-space reduction approach. Therefore, two new concepts regarding classifier's reliability in its predictions will be investigated and developed in connection with evolving fuzzy classifiers: conflict and ignorance. Conflict models the extent to which a new query point lies in the conflicting region between two or more classes. Ignorance represents the extent to which the new query point appears in an unexplored region of the feature space. The results based on real-world streaming classification data will show a stable high predictive quality of our approach, despite the fact that the requested number of class labels is decreased by up to 90%.