Mutual Knowledge-Distillation-Based Federated Learning for Short-Term Forecasting in Electric IoT Systems

As renewable energy resources increasingly integrate and dynamic user loads emerge, precise short-term forecasting across various scenarios becomes crucial for the efficient operation of modern power systems. Federated learning (FL) presents an effective solution for forecasting that safeguards privacy and security through model aggregation without sharing sensitive raw data. In the Electric Internet of Things (EIoT), the heterogeneity of clients can complicate model convergence, challenging the efficiency of the current federated averaging (FedAvg)-based methods. To address this, our analysis of regression challenges within the EIoT and the unique attributes of FL methods has led to the development of a novel algorithm: federated mutual knowledge distillation learning for regression (FedMR), aiming to deliver superior forecasting precision and efficiency at a reduced cost. FedMR utilizes the models and min-max labels from distributed clients, enabling its embedded generator to assimilate global knowledge and synthesize data in a normal distribution pattern. This feature enhances training across a spectrum of user environments. Demonstrated through testing on three open EIoT data sets, FedMR showcases high efficiency, accuracy, and adaptability across diverse client participation scenarios. Additionally, our research delves into the application of FedMR in managing device heterogeneity, a common issue in the EIoT systems. By focusing on sharing only the regression layers of clients, our proposed FedMRL(ightweight) pioneers a model-agnostic aggregation approach in short-term forecasting, effectively achieving a harmonized balance of accuracy and efficiency.