FedHTGL: A Two-Stage Federated Hetero-Task Graph Learning

The heterogeneity among clients poses a challenge to the effectiveness of the global model developed via federated learning. In this paper, we consider the heterogeneity task (Hetero-Task) graph federated learning scenarios. Each client executes different types of learning tasks, and the local dataset has different feature dimensions and label distributions in federated learning. In recent years, heterogeneous federated learning (hFL) and personalized federated learning (pFL) algorithms have emerged as effective methods to address the challenges posed by heterogeneous data. However, the existing hFL method can't simultaneously address both heterogeneous tasks and data. We propose FedHTGL (Federated Hetero-Task Graph Learning), a novel two-stage FL algorithm: model hierarchical aggregation optimization and knowledge distillation (KD)-based local fine-tuning. In particular, each local GNN network consists of a feature alignment layer, a feature representation layer, and a personalization task layer. In the first stage, the parameter sharing for the feature representation layer is performed to learn globally, while the alignment layer and personalization task layer undergo local optimization simultaneously. In the fine-tuning stage, unlike directly using local data for training. FedHTGL uses the model trained in the first stage as a teacher model to supervise the local model fine-tuning, preventing the local model from forgetting global knowledge. We conduct experiments on a Hetero-Task graph dataset, demonstrating that FedHTGL provides significant improvement in each local GNN model performance compared to state-of-art hFL or pFL.