Personalized Multi-task Federated Learning on Non-IID Medical Health Care Data

Medical institutions’ internal data is closely related to individual privacy information, and has a high degree of confidentiality and sensitivity in data storage and data use. Federated Learning (FL) is a distributed computing paradigm proposed to improve data privacy and quality issues in edge computing. One challenge in FL is data heterogeneity, which reduces training quality because the training data on local devices usually does not have the ideal Independent-and-Identically-Distributed (IID) property. The normalization layers in neural network structures play a key role in improving model accuracy in FL. In this work, the authors investigate the efficiency of applying group normalization (GN) instead of the commonly used batch normalization, aiming to improve the performance of the global model. The authors further identify a shortcoming of using GN, which is its degraded performance given the large batch size in training. Towards this end, they propose a federated learning system that features a normalization layer symGN, which introduces stochasticity into the GN model and improves its performance in the large-batch training scenario. A thorough evaluation of different non-IID datasets has demonstrated the efficiency of their proposed method. They use this personalized FL framework to protect privacy of medical data and share safe global models in different medical organizations.