Adaptive Federated Learning through Dynamic Model Splitting and Multi-Objective Clustering

Federated Learning (FL) enables multiple parties to collaboratively train models without centralizing data, making it ideal for privacy-sensitive applications. However, the heterogeneity and resource limitation of devices pose a critical challenge to the collaborative training process, incurring a significant communication cost to achieve convergence. Existing research has attempted to use clustering to address these issues. However, these approaches relied on a single clustering objective, limiting their effectiveness in a multifaceted heterogeneous environment. In this paper, we propose FedMSC, which employs an evolutionary-based multi-objective optimization approach to organize clients into distinct clusters via their similarities on independent factors such as response speed and local model updates. FedMSC iteratively generates Pareto-optimal cluster solutions, ensuring that no single solution outperforms another, while concurrently optimizing multiple objectives. Moreover, to account for computational diversity across clusters, FedMSC adopts a multi-exit training strategy in which the model is divided into blocks of layers, each equipped with auxiliary classifiers for early inference. Meanwhile, we devise a unique algorithm which dynamically assigns model blocks to devices through combinatorial optimization of devices’ resource capabilities and the computational requirements of the blocks. Experimental results demonstrate that FedMSC significantly reduce communication costs while maintaining a comparable accuracy to the baselines.