FedLC: Optimizing Federated Learning in Non-IID Data via Label-Wise Clustering

As contemporary systems are being operated in dynamic situations alternating into decentralized and distributed environments from conventional centralized frameworks, Federated Learning (FL) has been gaining attention for an effective architecture when aggregating the information and correspondingly trained AI model of geographically distributed datasets. However, the convergence of FL suffers from the skewed and biased non-IID local dataset acquired from the heterogeneous environment, which is common in real-world practice. To address this issue, we propose a novel Label-wise clustering algorithm in FL (FedLC) that guarantees the convergence among local clients that hold unique distribution. We theoretically analyze the non-IID attributes that potentially affect the performance, defining the six non-IID scenarios in hierarchical order. Through conducting experiments on the suggested non-IID scenarios, we empirically show that the vanilla FL aggregation model is incapable of gaining stable convergence, generating a biased model. By pre-evaluating the local dataset prior to training models, FedLC determines the potential contributions with respect to trainability in a global view, and adaptively selects the locals to collaboratively take part in while aggregation. Our experimental results show that the FedLC outperforms the state-of-the-art non-IID FL optimization studies, offering a robust convergence in a highly skewed and biased non-IID dataset.