DPFCIL: Differentially Private Federated Class-Incremental Learning on non-IID Data

Federated Learning (FL), as a distributed machine learning approach, makes it possible to realize global training and knowledge sharing of models while protecting data privacy. However, existing FL methods still face some challenges. First, since the local client model has only a limited parameter capacity when the model learns a new task or new data, previously learned information can be lost quickly and in large amounts, leading to catastrophic forgetting. Second, the private information may still be leaked by analyzing the parameters uploaded by the clients. Besides, the data distributions of FL clients are usually non-independent and identically distributed, leading to a degradation of the quality of the global aggregation model. These challenges pose obstacles to the effective implementation of federated learning. To address the above issues, we propose a novel differentially private federated class-incremental learning (DPFCIL) method. Specifically, it includes three aspects: 1) We design an elastic memory strategy that mitigates the class incremental model’s forgetting of old knowledge by dynamically expanding new modules and applying network regularization at the local client. 2) Before uploading parameters to the local client, we introduce differential privacy to prevent information leakage effectively. 3) We propose to use a dynamic adaptive aggregation method that relies on model parameter differences in the server aggregation phase to mitigate the loss of global accuracy due to the heterogeneity of the client’s data. Extensive experiments are designed to compare with state-of-the-art baseline methods on several datasets, and the experimental results show that DPFCIL outperforms other methods by an average of at least 5.6% in accuracy.