FL-Clip: Bridging Plasticity and Stability in Pre-Trained Federated Class-Incremental Learning Models

Federated learning (FL) is the prevailing paradigm in privacy-preserving machine learning. Despite recent advances yielding state-of-the-art outcomes, FL systems face challenges in adapting to dynamic real-world scenarios, where the local data distributions of clients may shift over time. This limitation stems from the common assumption of data stationarity in existing FL methods. In this paper, we propose the Federated Learning framework for CLass Incremental Pretrained models (FL-CLIP) to bridge this gap, by enabling continuous adaptation to changes in underlying FL client data distributions. The focus is on the incremental adaptation of existing FL models, trained on a substantial number of base classes, to newly arriving classes. To address the issue of catastrophic forgetting, the concepts of plasticity and stability are integrated into two distinct stages of FL-CLIP. Additionally, a task-weighted auxiliary loss is designed to tackle the class imbalance problem, and we propose a lightweight distillation method to enable FL-CLIP to adapt to resource-constrained environments. Extensive experiments demonstrate that FL-CLIP significantly outperforms 7 state-of-theart baselines in terms of average task accuracy, achieving a balanced performance across base and novel classes with a smaller memory footprint.