FedAttnPSO: An Attention-Guided PSO Algorithm for Enhancing Performance in Non-IID Federated Learning

Federated Learning (FL) is a distributed training paradigm designed to enable collaborative model training without sharing raw data. However, in practical settings, clients possess heterogeneous data in terms of quantity, distribution, and characteristics, leading to performance degradation in non-IID (Non-Independent and Identically Distributed) environments. To address this challenge, we propose FedAttnPSO, a novel federated optimization algorithm that integrates Particle Swarm Optimization (PSO) with an Attention Mechanism. FedAttnPSO selects the top-N clients with the lowest local loss values to prioritize those with superior model performance. After selecting top-N clients, the attention mechanism assigns dynamic aggregation weights to each selected client proportional to its contribution, allowing high-performing clients to have a greater influence while mitigating the impact of less effective ones. FedAttnPSO applies PSO to both local model updates on the client side and global updates on the server. This enables simultaneous optimization of local and global objectives, leading to more stable convergence. To evaluate the effectiveness of the proposed method, we conducted extensive experiments on the CIFAR-10 dataset under both IID and non-IID conditions, comparing FedAttnPSO against FedAvg and FedPSO. Experimental results show that FedAttnPSO achieves approximately $15 \%$ and $4 \%$ higher accuracy than FedAvg and FedPSO, respectively, under IID settings. In non-IID environments, it outperforms FedAvg by about $10 \%$ and FedPSO by approximately $26 \%$. These results demonstrate that FedAttnPSO can achieve more accurate and stable convergence in both IID and non-IID federated learning scenarios.