A Distributed First-Order Optimization Method for Strongly Concave-Convex Saddle-Point Problems

In this paper, we propose a first-order optimization method for solving saddle-point problems when the data is distributed over a strongly connected weight-balanced network of nodes. Our solution is based on the gradient descent-ascent where each node iteratively computes partial gradients of its local cost function to implement the corresponding steps. The proposed method further uses gradient tracking for both de-scent and ascent updates to tackle the local versus global cost gaps. We show that the proposed method converges linearly to the unique saddle-point when the global problem is strongly concave-convex. The numerical experiments il-lustrate the performance comparison of the proposed method with related work for different classes of problems.

1. Introduction and Related Work

Many applications related to signal processing, machine learning, and robust optimization naturally take the form of min-max problems [1]–[8]. The objective is to simultaneously minimize and maximize the cost function with respect to specific variables such that we find the point of equilibrium (or the saddle-point). In this paper, we consider the cost function : ; and assume that is convex in x and concave in y, where and . Thus, the objective is to find the saddle-point by minimizing with respect to x and maximizing with respect to y, i.e.,

$$\begin{equation*}\min_{\mathbf{x}\in \mathbb{R}^{p_{x}}}\max_{\mathbf{y}\in\mathbb{R}^{p_{y}}}F(\mathbf{x,y}).\end{equation*}$$