Robust Network Topologies for Distributed Learning

The robustness of networks against malicious agents is a critical issue for their reliability in distributed learning. While a significant number of works in recent years have investigated the development of robust algorithms for distributed learning, few have examined the influence and design of the underlying network topology on robustness. Robust schemes for distributed learning typically require certain conditions on the arrangement of malicious agents in the network. In particular, the majority of neighbors of any benign agent must be benign, and the subgraph of benign agents must be connected. In this work, we propose a scheme for the design of such topologies based on prior information of the risk profile of participating agents. We show that the resulting topology is asymptotically almost surely connected and benign agents have majority benign neighborhoods. At the same time, the proposed design asymptotically tolerates a fraction of malicious agents arbitrarily close to one, while risk agnostic designs, such as complete graphs, break down as soon as the majority of agents is malicious.