FedExg: Federated Learning with Model Exchange

Federated learning has successfully mitigated the privacy concerns for deep learning over distributed data, but suffers from performance decay led by the non-identical distribution of data. In this paper, we propose a novel framework, namely FedExg, to improve federated learning for non-IID data with model exchange. Two strategies, i.e., FedExg-S and FedExg-A, are developed to realize model exchange in a privacy-preserving fashion. FedExg-S randomly shuffles the models for broadcasting, while FedExg-A improves FedExg-S with partial aggregation to avoid model inversion attack. Theoretical analysis demonstrates that FedExg achieves a reduced upper bound of average regret in comparison to traditional model averaging methods. Experimental results show that FedExg improves image classification tasks on MNIST and CIFAR-10 dataset.