Fractional Position With Predictive Attention for Multivariate Time Series Forecasting

With the proliferation of the Internet of Things (IoT), a wealth of multivariate time series data is being generated across various domains, creating new demands for accurate and efficient forecasting models. Despite the success of attention-based models in capturing dependencies within time series, they often fail to address two critical challenges: 1) the lag effect between output and input, which can significantly distort predictions, and 2) the limitations of classic trigonometric positional embeddings, which lack scalability and adaptability to diverse temporal patterns. To address these challenges, we propose FPPformer, a novel forecasting model that introduces two key innovations: 1) a fractional positional embedding (FPE), which leverages fractional calculus to enable scalable and adaptive positional representations and 2) a predictive attention mechanism (PAM), which explicitly models the lag effect, aligning output and input more effectively. The FPPformer architecture consists of encoder-only structure, with the core of encoder module utilizing the PAM. Experimental results demonstrate that FPPformer significantly improves forecasting performance, reducing the mean squared error (MSE) by 28% and the mean absolute error (MAE) by 17% across six datasets spanning four domains—electricity, weather, economy, and transportation—especially on large-scale datasets such as Traffic and Electricity. These results highlight FPPformer’s ability to address fundamental challenges in time series forecasting, providing a new perspective on leveraging positional representations and lag-aware attention mechanisms. The code for this project is available at https://github.com/jancely/FPPformer.