Distributed Stochastic Averaging Gradient Algorithm over Directed Graph

This paper concentrates distributed stochastic opti-mization over directed network topology. In the network there is a collection of nodes that can access to their own local objective functions, and their target is to cooperate to minimize the sum of objective functions over multi-agent networks with directed network topology. For the large scale machine learning problems, the local objective functions are further assumed to be an average of a finite set of functions. We introduce a directed decentralized double stochastic averaging gradient method(DDSA) which utilizes the renowned push-sum protocol and stochastic averaging gradient to accelerate convergence in directed graph. DDSA can linearly converge in expectation to the optimal solution. We provide a numerical experiment based on logistic regression to illustrate the validity of DDSA.