Robust Peer-to-Peer Federated Learning with Deep Reinforcement Learning Based Client Selection Against Data Poisoning Attacks

While existing strategies offer partial solutions, enhancing robustness in decentralized, peer-to-peer FL remains a critical challenge due to the lack of centralized oversight, the variability of client data quality, and the dynamic nature of client participation. In this paper, we propose a deep reinforcement learning (DRL) method for client selection in peer-to-peer FL, enhancing model robustness by choosing reliable peers. By framing client selection as a Markov Decision Process (MDP), our approach enables clients to identify trustworthy peers, effectively mitigating data poisoning attacks and improving the accuracy and resilience of the aggregated model. Our proposed method models client selection as an MDP, where each client’s state is derived from condensed representations of other peers’ models’ parameters. The clients leverage Deep Q-Networks (DQN) to adaptively refine peer selection, ensuring robustness against malicious behaviors. Our experimental configuration involved training a convolutional neural network on CIFAR-10, executing different types of data poisoning attacks. Our results, averaged across the accuracies from all clients, demonstrate that the proposed method consistently achieves 5-7% higher accuracy compared to baseline approaches without sophisticated selection mechanisms, and surpasses the state-of-the-art CBE3 algorithm by 2-3%. Furthermore, our method exhibits significantly more stable accuracy trends and reduced noise levels throughout training, indicating enhanced resilience and adaptability under dynamic and adversarial conditions.