Distributed Online Optimization Of Consensus-Based Adaptive Gradients Over Time-Varying Networks

The application of distributed optimization to ICPSs has advantages and challenges. Recently, a consensus-based distributed adaptive moment estimation method, referred to as DAdam (Distributed Adam), has been proposed as a variant of Adam specifically tailored for distributed and parallel computing environments. DAdam integrates the benefits of adaptive learning rate and moment estimation, but its application scenarios are limited. The assumption of static networks in existing literatures is conservative in real environments. To overcome this limitation, we propose DAdam-TV which can solve the optimization problems in time-varying networks. After rigorous analysis, we exam the convergence of proposed algorithm. For convex and non-convex problems, we bound the dynamic regret and local regret, respectively. Numerical simulations show that DAdam-TV has better performance in solving optimization problems in dynamic networks. DAdam-TV breaks through the limitation of static application scenarios, which makes the algorithm more general and effective in practical applications such as ICPSs.