Corrected Gradient Methods for Distributed optimization

We propose a class of algorithms as Corrected Gradient Methods (CGMs) to solve the distributed optimization problem of minimizing an aggregate cost with consensus requirement on individual decision variables. Specifically, we design the algorithm by combining estimation and correction of the disagreement of individual gradients, and apply it respectively to gradient descent method and heavy ball method. With a doubly stochastic weight matrix over a strongly connected network, we show that in both cases CGM can achieve the convergence to a common optimum for L-smooth convex functions; if the function is also strongly convex, then the convergence is exponentially fast. Simulation is conducted to validate the efficacy of CGMs.