Fast-Converging Decentralized ADMM for Consensus Optimization

For its well-established convergence properties and applicability to various optimization problems, the alternating direction method of multipliers (ADMM) has been at the center of several research fields. When applied to distributed problems such as consensus optimization, ADMM is typically implemented in a centralized manner. Such implementations are, however, discouraged for e.g. their dependency on the location and capacity of the central node. While there are decentralized alternatives, these implementations are either computationally and communication-wise expensive or slow. This is because existing decentralized alternatives require all worker nodes to either replicate the work of synchronizing the outputs from all nodes or execute their tasks in sequence. To address this problem, we propose a fast-converging decentralized ADMM (FCD-ADMM) algorithm. Through theoretical analysis, we prove the convergence properties of FCD-ADMM and show that FCD-ADMM can converge faster than its centralized alternative without sacrificing accuracy. As shown in our numerical experiments, FCD-ADMM can converge to the same or better solution faster than several state-of-the-art alternatives.