Enhancing Federated Learning With Pattern-Based Client Clustering

The exponential growth of the Internet of Things (IoT), driven by the increasing number of connected devices and sensors, is profoundly transforming many industries. This proliferation has been enabled by advances in storage and processing capabilities, facilitating the implementation of deep model. However, concerns surrounding privacy and substantial bandwidth required for data transmission arise from the centralized processing of data collected from numerous distributed devices. To address these issues, federated learning (FL) has emerged as a viable solution with a decentralized strategy. It ensures data privacy by allowing devices to locally train models while keeping their data localized, thus reducing centralization needs and concerns related to privacy and bandwidth. To mitigate the data heterogeneity problem in FL scenarios, we propose federated pattern extraction clustering (FedPEC) to cluster clients with similar data distributions. Compare to existing client clustering methods, FedPEC requires no additional data transmission and ensures great flexibility, scalability and privacy. Furthermore, we discuss the discrepancy between representation and data distribution in existing methods from the perspective of pattern representation. Based on this, we propose a variant of FedPEC as a solution, which has achieved excellent performance on multiple FL data sets.