Representation Robustness and Feature Expansion for Exemplar-Free Class-Incremental Learning

Despite deep neural networks have made outstanding achievements in many static tasks, when faced with a continuous stream of data, they suffer from catastrophic forgetting since the previous data is usually inaccessible. Stored data or generative model is commonly used for maintaining the model performance but with memory utilization and privacy safety issues. Prototype-based methods address these issues by keeping only one prototype for each class but with limitations in its ability to trade-off the model stability and plasticity. In this paper, a novel exemplar-free class-incremental learning method is proposed which improves the stability of the representation learning and the decision boundary to a great degree. First, based on the results of our exploration into the impact of the batch normalization (BN) layer on representation learning, we propose to remove the BN layer (RBNL) in the incremental training phase to improve the stability of model representation learning. Then, to further maintain the feature space, we design the prototype mixing (PM), which expands the deep features by randomly and linearly combining prototypes of the old classes to generate hybrid prototypes with composite labels for fine-tuning the fully connected layer. Experimental results on three benchmark datasets, CIFAR-100, TinyImageNet, and ImageNet, show that our proposed method can effectively balance the stability and plasticity of the model, and outperforms the state-of-the-art works.