Extending Control Theory into Federated Learning Data Heterogeneity Problem

Federated Learning (FL) offers a collaborative and privacy-preserving distributed machine learning framework. However, statistical heterogeneity is one of the key challenges that affect the performance of an FL process. Previous solutions like SCAFFOLD and FedProx have made strides in improving FL's accuracy and convergence speed by addressing this heterogeneity. This work aims to reframe the data heterogeneity problem into a control theory problem. After reviewing various aggregation algorithms, we establish a connection between FL and control theory, leading to the development of the SCAFFOLD-PID algorithm. Which extends the SCAFFOLD algorithm by incorporating PID control theory elements. We present empirical results showcasing the performance of SCAFFOLD-PID, providing insights into its efficacy and potential in FL environments.