Joint Focal Loss and Dominant Gradient Correction for Gradient Conflict in Federated Learning

Federated learning faces significant challenges of data heterogeneity among clients. In this heterogeneous data scenario, the preference of local models can result in gradient conflict during the global aggregation phase. This can significantly lead to a decrease in the global accuracy. In this paper, we present a novel Federated Learning Mitigating Gradient Conflict method named FedMGC that aims to mitigate gradient conflict. FedMGC replaces the cross-entropy loss function with the focal loss function. This balances the proportion of each class in the loss and reduces the preference of the local models for majority classes. In the global aggregation phase, we design the dominant gradient correction method called DGC to improve the global accuracy. Specifically, we select some of the gradients with small outliers to form the dominant gradients. And then we use dominant gradients to adjust the local gradients and alleviate the gradient conflict. In the evaluation experiments on heterogeneous datasets, FedMGC achieves higher test accuracy compared to baselines. In particular, over the CIFAR-10 dataset, FedMGC achieves 3.88%, 1.7% and 1.31% higher test accuracy than those of FedAvg, FedProx and FedNova, respectively.