FedMAR: A Privacy-Preserving and Robust Server-Side Multi-Stage Federated Learning

In recent years, federated learning (FL) has continued to evolve with the advent of big data and the large language model (LLM), but it has also exposed numerous security and privacy issues. As a form of distributed machine learning, FL systems are more susceptible to poisoning attacks because training data are dispersed across different participants; additionally, the training achievement of FL may be subject to low-cost theft by some free-riders. Existing works have addressed defenses against the aforementioned two types of threats, but they often focus on defending against only one type and fail to effectively integrate defenses against multiple types of threats. However, in real-world Internet of Things (IoT) systems, the types of threats are not limited to just one category. In this work, we try to maintain the performance of the global model under poisoning attacks, preserve the privacy of the server under free-riders, and explore the balance between these two aspects. Therefore, this work proposes Federated Multi-Stage Asynchronous Roll-back (FedMAR), ensuring the quality of local updates; in addition, this work also provides privacy preservation in the global update process based on Rinyi Differential Privacy (RDP), and offers a certain basis for detecting free-riders. To validate the generalization of the proposed method, we conducted relevant experiments on both image and text datasets, and further investigated the robustness of the proposed method against poisoning attacks, model inversion attacks, data heterogeneity, and other aspects. The testing accuracy of the global model can even be improved by 7.2%.