GLRB: Heterogeneous Federated Continual Learning via Global and Local Rebalance

Federated continual learning (FCL) as an emerging learning paradigm empowers federated learning (FL) to continuously learn new tasks, allowing it to adapt to dynamic scenarios. Despite many FCL studies have emerged, most of them focus on the catastrophic forgetting and neglect the challenges brought by heterogeneity, which hinders FCL methods from being applied to real-world scenarios. FCL presents new challenges in both data and model heterogeneity compared to FL, including data heterogeneity in time and space dimensions and model heterogeneity in output and weight spaces. We propose a novel approach called global and local rebalance (GLRB) to address the challenges of model and data heterogeneity. We propose a unified feature space for knowledge exchange in the presence of model heterogeneity, and two communication strategies are proposed to accommodate different communication conditions We demonstrate the effectiveness of GLRB through comparison and ablation experiments on multiple datasets.