TravellingFL: Communication Efficient Peer-to-Peer Federated Learning

Peer-to-Peer federated learning is a distributed machine learning paradigm with a primary goal of learning a well-performing global model by collaboratively learning a shared model at different data hubs without the need of sharing data. Due to its immense practical applications, there is growing attention towards various challenges of efficient federated learning including communication efficiency, assumptions on connectivity, data heterogeneity, enhanced privacy, etc. In this paper, we address the problem of dynamic network topologies in federated learning. We present a technique to help new participants in Peer-to-Peer federated learning reach best possible accuracy by leveraging learning at other devices in a communication efficient manner. We model the costs in federated learning and apply a graph theoretical framework to show that one can draw from a range of graph-based algorithms to construct an efficient communication algorithm on a connected network, thereby matching the inference efficiency of centralized federated learning. We conduct experiments with varied graph formations and sizes to validate our claims.