MSGFormer: Revolutionizing Traffic Flow Prediction With Multi-Scale and Gated Transformer Architecture

Traffic flow prediction has emerged as a critical component in advancing smart cities. Nevertheless, precisely forecasting traffic flow remains a formidable challenge, attributable to the intricate and dynamic spatiotemporal interdependencies inherent within traffic data. Contemporary methodologies often overlook the different impacts exerted at each timestamp and treat temporal correlations, rendering them incapable of extracting temporal patterns at multiple scales. Furthermore, these approaches fail to account for the neighboring and functional relationships among nodes within the spatial module. In this work, we introduce an innovative Multi-Scale and Gated transFormer architecture, MSGFormer, to overcome the inherent limitations for accurate traffic flow estimation. We have introduced a multi-scale sampling strategy wherein we sample from the original data at three scales: recent, daily, and weekly. This approach enables the generation of corresponding subsequences that encapsulate temporal information across different granularities. Subsequently, each generated subsequence is projected into a latent space and systematically combined with positional, temporal, and spatial embeddings. The positional embedding comprises relative positional embedding and time stamp embedding corresponding to days and weeks, aiming at capturing the sequential and cyclical characteristics of the data. Furthermore, at the inception of the transformer encoder, a gated unit, composed of a neighboring mask and a functioning mask, is employed to capture both static and dynamic spatial correlations effectively. Comprehensive experiments have been conducted on four real-world traffic datasets. The experimental results robustly validate that our model attains significantly higher predictive accuracy in comparison to other baseline models.