Kullback-Leibler Reservoir Sampling for Fairness in Continual Learning

Continual Learning (CL) aims to update a ML model with continuously arriving training data without having to retrain the model again from scratch. Continual Learning will play a critical role in future AI-enabled services at the network edge. We propose an online replay-based Continual Learning method for classification problems, in which the learner stores data points to a buffer and replays them during training. The core of our contribution is a new replay buffer content update policy that selects data points to store in the buffer by minimizing the Kullback-Leibler (KL) loss between the buffer distribution and a target distribution. This results in a new algorithm that generalizes previous replay-based methods such as Reservoir Sampling and Class-Balancing Reservoir Sampling. We set the target distribution to be dependent on the evolving empirical distribution of classes in the training data stream, and we parameterize it with a single parameter. This allows to model different target class distributions in the buffer such as the class distribution of the training data stream, the uniform class distribution, and a distribution with class percentages that are inversely proportional to those in the stream. We show that the KL-based approach improves the fairness in multi-class imbalanced data streams by resulting in more similar per-class test accuracy scores among all classes. We apply our method to several Continual Learning problems using the MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets, and we show that our method is better than the Reservoir Sampling and Class-Balancing Reservoir Sampling schemes in terms of class fairness and overall accuracy.