FedLCA: Synchronous Layer-Wise Compensated Aggregation for Straggler Mitigation

In this paper, we introduce a layer-wise compensated aggregation algorithm designed for federated learning in dynamic environments. To reduce data transmission and maintain model continuity, our method implements a hierarchical update strategy that avoids artificially modular partition of neural networks and thus more flexible in practice. Specifically, weight layers closer to the output, which significantly impact model performance, receive direct updates. Conversely, layers further from the output with minimal influence, are updated using gradient compensation from the previous iteration. To ensure consistency across the model, we compensate these less influential weight layers with residual values from earlier rounds, thus enhancing integration and continuity. Experimental results from two datasets confirm that our algorithm not only secures stable convergence but also matches the generalization performance of traditional federated averaging, even under conditions with dropout rates as high as 90%.