Active Learning for Imbalanced Domains: the ALOD and ALOD-RE Algorithms

Active learning strategies are used to acquire an enlarged labelled training set that allows the learner to achieve a better performance. To this end, unlabelled instances are carefully selected for labelling by an human expert in order to achieve the best performance with the smallest number of questions to this oracle. Several techniques exist to select the most informative samples within active learning strategies. However, the effectiveness of these methods when applied to problems with imbalanced classes was not studied before. In this paper, we focus on the improvement of instance selection strategies for active learning techniques in the presence of class imbalance. In an imbalanced setting, learning algorithms have difficulties to focus on the minority class due to its under-representation. Still, this is typically the class of interest for the end-user. This mismatch between the classes distribution and the goals of the end user is known as the class imbalance problem. In an active learning setting for class imbalance problems, this becomes an even more challenging issue. We propose two novel active learning algorithms, ALOD and ALOD-RE centered around selecting the most informative samples to be labelled, while considering the selection of possible minority class cases and the generation of synthetic minority class examples to improve the learner performance. To this end, our two proposed solutions combine: active learning procedures, resampling strategies, and anomaly detection methods. Through an extensive set of experiments we show that the incorporation of outlier detection and resampling techniques in the active learning procedure benefits the learners performance on imbalanced domains. The performance advantages of our proposed ALOD and ALOD-RE algorithms are clearly supported by our experimental results.