Loading [a11y]/accessibility-menu.js
Accelerating Wireless Federated Learning via Nesterov’s Momentum and Distributed Principal Component Analysis | IEEE Journals & Magazine | IEEE Xplore

Accelerating Wireless Federated Learning via Nesterov’s Momentum and Distributed Principal Component Analysis


Abstract:

A wireless federated learning system is investigated by allowing a server and multiple workers to exchange uncoded information via orthogonal wireless channels. Since the...Show More

Abstract:

A wireless federated learning system is investigated by allowing a server and multiple workers to exchange uncoded information via orthogonal wireless channels. Since the workers frequently upload local gradients to the server via band-limited channels, the uplink transmission from the workers to the server becomes a communication bottleneck. Therefore, a one-shot distributed principal component analysis (PCA) is leveraged to reduce the dimension of uploaded gradients to relieve the communication bottleneck. A PCA-based wireless federated learning (PCA-WFL) algorithm and its accelerated version (i.e., PCA-AWFL) are proposed based on the low-dimensional gradients and the Nesterov’s momentum. For the non-convex empirical risk, a finite-time analysis is performed to quantify the impacts of system hyper-parameters on the convergence of the PCA-WFL and PCA-AWFL algorithms. The PCA-AWFL algorithm is theoretically certified to converge faster than the PCA-WFL algorithm. Besides, the convergence rates of PCA-WFL and PCA-AWFL algorithms quantitatively reveal the linear speedup with respect to the number of workers over the vanilla gradient descent algorithm. Numerical results are used to demonstrate the improved convergence rates of the proposed PCA-WFL and PCA-AWFL algorithms over the benchmarks.
Published in: IEEE Transactions on Wireless Communications ( Volume: 23, Issue: 6, June 2024)
Page(s): 5938 - 5952
Date of Publication: 08 November 2023

ISSN Information:

Funding Agency:


References

References is not available for this document.