A Scalable Secure Fault Tolerant Aggregation for P2P Federated Learning

In federated learning (FL), client models rather than the training data are uploaded to the aggregation server, so that clients can participate in a learning process while maintaining their privacy. However, there is still a possibility that the client's data can be inferred from the aggregated model. Also, when the aggregation server fails, the learning process is interrupted. P2P FL systems that eliminate a single point of failure or enable secure aggregation have also been proposed, but they have scalability issues due to the high communication cost. In this paper, we therefore propose a scalable, secure and fault tolerant aggregation system for P2P FL. By introducing a two-layer network which consists of the Secure Average Computation (SAC) layer and Federated Averaging (FedAvg) layer, our system achieves both the privacy protection of participating peers and the reduction of total communication cost. We also propose a fault-tolerant SAC to handle peer dropouts during aggregation. As a backend to the above system, we propose a two-layer Raft, which successfully improves the fault tolerance against random peer crashes. Experimental results show that our two-layer aggregation system outperforms the original SAC in terms of the communication cost and fault tolerance with the comparable accuracy, slow subgroups in SAC layer do not affect the overall accuracy and our two-layer Raft maintains availability by quickly detecting a crashed leader and replacing it with new one. The analysis shows that our system consisting of 30 peers reduces communication costs by 10.36x while providing fault torelance.