A Robust Framework for Fixing The Vulnerability of Compressed Distributed Learning

Nowadays, as a prevailing paradigm for large-scale machine learning, distributed learning has been faced with two challenges, communication bottleneck and limited robustness. For the communication challenge, compression is widely-used as a solution. For the robustness challenge, some robust defence methods have been proposed. However, previous works that simultaneously consider these two challenges are limited. Through an experiment on the w8a dataset, we found that compressed distributed learning with rand-K is vulnerable to poisoning attacks. Therefore, in this paper, we propose a robust compressed distributed framework for distributed learning settings. Experimental evaluations on a9a and w8a datasets have shown the effectiveness of our proposed framework, which markedly decreases the average optimality gap from 1.47E −2 and 2.15E −2 to 3.98E −4 and 4.33E −4 respectively.