Decentralized P2P Federated Learning on Ad-hoc Like Networks with Non-IID Dataset

In the last few decades, Federated Learning (FL) is proposed in order to perform Machine Learning (ML) tasks in a distributed manner while protecting users' privacy and data. However, most of the traditional FL methods rely on centralized entities while in many real-life situations, there isn't any central server which can orchestrate the training procedure. Moreover, it is also easy to be ignored by researchers that the network topology is likely to be changing all the time in some scenarios such as Ad-hoc networks. Besides, how to deal with the unbalanced data which are not independently identically distributed (IID) collected by devices is also an important open problem. As a consequence, in this paper, we propose a peer-to-peer federated learning algorithm with ad-hoc network, along with five model aggregation strategies. We tested our algorithm on self-made Non-IID datasets. After 5000 epochs, the average accuracy of devices reaches 69% under the best strategies on unbalanced CIFAR10 dataset, improving 28% from the self-training cases without P2P communication. The results indicate that our strategies can reduce the negative effect caused by Non- IID datasets even with ad-hoc networks.