From Routine to Reflection: Pruning Neural Networks in Communication-efficient Federated Learning

Communication-efficient federated learning benefits from neural network pruning, as it speeds up training and reduces model size. However, existing pruning techniques may not be optimally suited for joint model training in federated learning, particularly regarding the choice of pruning sites and the sequence of pruning operations. In this work, we explore server-side pruning using a similarity-based approach, contrasting it with methods that prune on either the clients or the server. Our method uses global model parameters to calculate the difference between global and local models, guiding the pruning process. We also examine how the order of pruning affects performance. Experimental results show that our method maintains model performance in non-IID settings while reducing communication overhead by 50%-70%. To simulate a realistic FL setup, we run server-side pruning on a CPU, increasing CPU involvement, distributing client workloads, and reducing energy consumption.