Dictionary learning over large distributed models via dual-ADMM strategies

We consider the problem of dictionary learning over large scale models, where the model parameters are distributed over a multi-agent network. We demonstrate that the dual optimization problem for inference is better conditioned than the primal problem and that the dual cost function is an aggregate of individual costs associated with different network agents. We also establish that the dual cost function is smooth, strongly-convex, and possesses Lipschitz continuous gradients. These properties allow us to formulate efficient distributed ADMM algorithms for the dual inference problem. In particular, we show that the proximal operators utilized in the ADMM algorithm can be characterized in closed-form with linear complexity for certain useful dictionary learning scenarios.