AMbit: An Efficient Pruning Technique in Federated Learning for Edge Computing Systems

The Industrial Internet of Things (IIoT) has revolutionized industrial sectors with enhanced connectivity, data exchange, and predictive maintenance. However, it faces various challenges from non-IID data distributions and communication overheads, to the consistency and privacy of prediction models for maintenance. Federated Learning (FL) has been considered as a prevalent technique to address privacy concerns. On the other hand, Edge computing (EC) is being increasingly introduced to ensure low-latency data processing in IIoT systems, especially those with time-critical requirements, e.g., industrial robotics and motion control systems. Moreover, the complexity and design dimensions of today’s IIoT systems has lead to the development of large Machine Learning (ML) models with millions of parameters, e.g., using computer vision for field management. This introduces computational and privacy challenges in IIoT scenarios. Innovative FL optimization approaches such as pruning are aimed to tackle these challenges by reducing the number of training parameters, while they may impact accuracy. In this paper, we propose a new technique, called Adaptive Mean aBsolute devIaTion (AMbit), which is an innovative pruning approach optimizing data transmission without compromising model accuracy and inducing additional computation overhead. By dynamically comparing the difference between the current and the previous weight value, AMbit adapts better to parameter fluctuations at different stages, thereby accurately locating those parameters that have less impact on convergence. AMbit’s generality and efficiency are illustrated using MNIST and CIFAR-10 datasets, outperforming traditional Magnitude pruning in FL. For MNIST, AMbit reduces data uploads up to 43.75%, with an increase of 0.62% in accuracy. While for For CIFAR-10, AMbit achieved a 63.44% decrease with a 5.22% drop in accuracy.