Dynamic Client Distillation for Semi-supervised Federated Learning in A Realistic Scenario

Recent advancements in semi-supervised federated learning (SSFL) have significantly enhanced public health services by enabling medical institutions to share model updates via a central server. However, most SSFL approaches are based on conservative assumptions, such as labels-at-server and labels-at-client, which fail to fully capture the complex and diverse data distributions inherent in medical institutions. To address this limitation, we introduce a novel application of SSFL tailored to a realistic client data scenario, encompassing clients with fully-labeled, partially-labeled, and fully-unlabeled data. This approach effectively navigates varying levels of data annotation by maximizing the utility of unlabeled samples within the client federation. To tackle the challenges posed by such a complex scenario, we propose a new SSFL framework, FedCD. FedCD incorporates three client-distilled models, each corresponding to a distinct client data distribution, alongside server-client federation. First, each client-distilled model condenses the diverse parameters of the client federation into robust knowledge through distillation. The contribution of each client model is then dynamically adjusted based on its proximity to the client-distilled model, ensuring that the framework adapts to the heterogeneous characteristics of individual clients. By aggregating client-distilled models, FedCD implements model drift correction, effectively mitigating parameter drift across heterogeneous models. This dynamic federated approach not only harnesses unlabeled data efficiently but also accommodates diverse annotation levels while adapting to varying data distributions. Extensive experiments on two medical image segmentation tasks and one classification task demonstrate the superiority of our method, highlighting its ability to address realistic challenges in medical data scenarios.