Toward Resource-Efficient Federated Learning in Mobile Edge Computing

Federated learning is a newly emerged distributed deep learning paradigm, where the clients separately train their local neural network models with private data and then jointly aggregate a global model at the central server. Mobile edge computing is aimed at deploying mobile applications at the edge of wireless networks. Federated learning in mobile edge computing is a prospective distributed framework to deploy deep learning algorithms in many application scenarios. The bottleneck of federated learning in mobile edge computing is the intensive resources of mobile clients in computation, bandwidth, energy, and data. This article first illustrates the typical use cases of federated learning in mobile edge computing, and then investigates the state-of-the-art resource optimization approaches in federated learning. The resource-efficient techniques for federated learning are broadly divided into two classes: the black-box and white-box approaches. For black-box approaches, the techniques of training tricks, client selection, data compensation, and hierarchical aggregation are reviewed. For white-box approaches, the techniques of model compression, knowledge distillation, feature fusion, and asynchronous update are discussed. After that, a neural-structure-aware resource management approach with module-based federated learning is proposed, where mobile clients are assigned with different subnetworks of the global model according to the status of their local resources. Experiments demonstrate the superiority of our approach in elastic and efficient resource utilization.

Introduction

Recently, federated learning has attracted a lot of attention as a novel distributed deep learning paradigm [1]. In federated learning, the clients separately train their local deep neural network (DNN) models with their private data. The local model updates are then sent to the central server, while the private data remain on the clients. After collecting all local updates, the central server is responsible for aggregating a new global model, which will be delivered to the clients for the next round of model training. This distributed training iteration repeats until the global model converges to a satisfying test accuracy. By using federated learning, the individual privacy of clients could be effectively preserved as no private data is shared among the clients and the central server.