Hybrid Transformer-KAN Within Federated Learning Framework: A Novel Machine Learning Approach for Improved Short-Term Weather Forecasting

Among the various methods used for time series prediction tasks in weather forecasting, Transformer can effectively extract global information and has achieved tremendous success. However, Transformer is challenged in forecasting series with large lookback windows due to performance degradation and computation explosion. At the same time, a new architecture of Kolmogorov–Arnold Network (KAN) has been proposed, which dramatically improves the fitting ability in constructing neural networks widely used in machine learning techniques. In order to take both advantages of Transformer and KAN, in this paper a novel method of Transformer-KAN is proposed. To be specific, the Transformer is used to extract global features and pass them to KAN, and the latter is responsible for learning global features and optimizing them with high precision. With a deep integration of Transformer and KAN, the proposed Transformer-KAN exhibits powerful feature extraction capability and fitting ability. Furthermore, federated learning is applied to the proposed method to exploit the spatio-temporal dependencies while protecting data privacy. As a result, a novel framework of Federated-Transformer-KAN is proposed to collaboratively forecast weather variables in a distributed manner, without sharing data that is collected by the intelligent sensors equipped at each weather station locally. Compared with the benchmark methods, extensive experimental results shown that better prediction accuracy and generalization are achieved by the proposed Federated-Transformer-KAN. For instance, in the task of predicting the next 1 ambient temperature at station 68328, the mean squared error of Transformer and KAN individually is reduced by 33.9% and 40.9%, respectively.