Distributed Stochastic Strongly Convex Optimization under Heavy-Tailed Noises

The distributed optimization problem is examined in this study while heavy-tailed gradient noises are present. A heavy-tailed noise ξ indicates that the noise may not always satisfy the bounded variance requirement. As an alternative, it meets a broader assumption. The Pareto distribution noise is a common example of this type of noise. A number of distributed optimization techniques have been developed for the heavy-tailed noise situation; however, these algorithms require a centralized server within the network to gather client data. A distributed method using gradient clipping is proposed; unlike these algorithms, it takes into account the absence of a centralized server.