Frequency-Domain Analysis of Distributed Optimization: Fundamental Convergence Rate and Optimal Algorithm Synthesis

The design of optimization algorithms has long been a matter of art, and it calls for systematic development of methods in which algorithms can be analyzed, designed, and benchmarked with regard to their key attributes such as efficiency, complexity, and robustness. This paper answers to this need by providing a frequency-domain framework for algorithm analysis and synthesis, with a particular emphasis on distributed optimization problems. We propose a general class of gradient-based distributed algorithms that can be viewed as interconnected dynamical systems consisting of a linear time-invariant subsystem and a Lur'e-type nonlinear component, thereby enabling analysis and synthesis of such algorithms from a robust control perspective via the usage of the circle criterion and the Zames–Falb theorem. By linking the convergence of an algorithm with the absolute stability of a corresponding Lur'e system, optimization of the algorithmic convergence rate is recast as a Nevanlinna–Pick interpolation problem akin to an
$\mathcal {H}_{\infty }$
optimal control problem. Solutions to such interpolation problems lead to a variety of gradient-based algorithms with optimal and suboptimal convergence rates, and algorithmic complexity judiciously controlled.