Federated Knowledge Distillation using Hierarchical Reinforcement Learning in Resource-Constrained IoT Edge-Cloud Computing Environments

With the development of Federated Learning (FL) in IoT Edge-Cloud Computing environments, mobile terminals are able to cooperate without the leakage on raw data. However, factors including the terminals' high mobility and the network fluctuations make the cooperator selection during FL training extremely complex. Under the distributed cooperation, traditional FL strategies show certain limitations and cannot always select the available nodes when training, leading to the difficulties in energy and latency optimization. In this paper, we propose a Hierarchical Reinforcement Learning (HRL)-based federated knowledge distillation (HRL-FedKD) framework in which both high-level and low-level controllers utilize the Double Deep Q-Network (DDQN) algorithm. The high-level controller selects the nodes participating in FL training, while the low-level controller determines the number of local training epochs for each node. After training, the global model will be compressed into a lightweight model by knowledge distillation (KD) in deployment while preserving the personalization of local models. The experiments were conducted using Chest X-Ray and Brain Tumor MRI datasets to validate the proposed FL strategy. The results demonstrate that the HRL-FedKD framework can effectively optimize latency and energy consumption in complex state spaces.