Shuffle Private Decentralized Convex Optimization

In this paper, we consider the distributed local stochastic gradient descent (SGD) algorithm by parallelizing multiple devices in the setting of stochastic convex optimization (SCO). The losses in the majority of the earlier literatures are required to satisfy Lipschitzness and smoothness, and the privacy leakage may exist in the calculation of gradients. Hence, by incorporating the Hölder smooth loss and the shuffle model of differential privacy (DP) to study the local SGD algorithm in parallel multiple distributed devices, we proposed a distributed learning of local SGD with sequentially interactive shuffle private algorithm (Shuffle-DSGD) under equal intercommunication interval scheme. We provide the privacy guarantees by using advanced composition and shuffle protocol for vector summation. We also analyze the convergence bound of the Shuffle-DSGD and obtain the optimal excess population risk
$\mathcal {O}(1/T)$
up to logarithmic factors with gradient complexity
$\mathcal {O}(n)$
. It turns out that our convergence rate is superior to the one
$\mathcal {O}(1/\sqrt {T})$
in the existing work and both of the gradient complexities are consistent. The effectiveness of our algorithms is demonstrated by synthetic and real datasets.