I Got Your Back: Efficient Federated Learning with Backpropagation-Free Training

In Federated Learning (FL), stragglers refer to the slow devices with delayed updates that stall the model aggregation at the server. Because of the inevitable system heterogeneity, low-end devices easily become stragglers, which are usually excluded from training for training efficiency, despite the fact that they may possess valuable data for the training of the global model. To overcome this challenge, we design FedForward, a FL architecture that replaces expensive backpropagation with forward gradients on low-end devices, so that low-end devices may update their local models based on the global model changes from the previous communication round. With FedForward and its enhancement, FedForward-PM, training time on low-end devices can be greatly reduced, without sacrificing the performance of the global model. Experiments show that, compared to FedAvg, FedForward significantly reduces computation time (52.32% and higher) and computational resource consumption (up to 77.78%) for low-end devices, in both IID and non-IID scenarios. In a practical sense, FedForward permits low-end devices to be included as much as possible in FL training, and is hence especially meaningful for FL deployments in IoT environments.