A Lightweight and Accurate Spatial-Temporal Transformer for Traffic Forecasting

We study the forecasting problem for traffic with dynamic, possibly periodical, and joint spatial-temporal dependency between regions. Given the aggregated inflow and outflow traffic of regions in a city from time slots 0 to
$t - 1$
, we predict the traffic at time
$t$
for any region. Prior arts in the area often considered the spatial and temporal dependencies in a decoupled manner, or were rather computationally intensive in training with a large number of hyper-parameters which needed tuning. We propose ST-TIS, a novel, lightweight and accurate
S
patial-
T
emporal
T
ransformer with
i
nformation fusion and region
s
ampling for traffic forecasting. ST-TIS extends the canonical Transformer with information fusion and region sampling. The information fusion module captures the complex spatial-temporal dependency between regions. The region sampling module is to improve the efficiency and prediction accuracy, cutting the computation complexity for dependency learning from
$O(n^{2})$
to
$O(n\sqrt{n})$
, where
$n$
is the number of regions. With far fewer parameters than state-of-the-art deep learning models, ST-TIS's offline training is significantly faster in terms of tuning and computation (with a reduction of up to
$90\%$
on training time and network parameters). Notwithstanding such training efficiency, extensive experiments show that ST-TIS is substantially more accurate in online prediction than state-of-the-art approaches (with an average improvement of
$9.5\%$
on RMSE, and
$12.4\%$
on MAPE compared to STDN and DSAN).