Nys-FL: A communication efficient Federated learning with Nyström approximated Global Newton direction

High communication overhead is a challenging problem in Federated Learning (FL), which needs to be minimized. One way to reduce the overall communication overhead is directed to fast optimization methods, which can reduce the number of communication rounds by improving the convergence of the global model. This paper introduces Nys-FL, which aims to accelerate the convergence of federated learning by incorporating global data distribution and Nystrom approximated global Newton direction, while updating the global model. Nys-FL uses federated Gaussian mixture models (FedGMMs) to find the global data distribution. Extensive experiments on several heterogeneously partitioned datasets show that Nys-FL significantly outperforms existing state-of-the-art federated algorithms such that FedProx, SCAFFOLD, GIANT and DONE in term of reduced communication rounds, while achieving a certain precision of the global model's convergence.