Spatial-Temporal Interval Aware Individual Future Trajectory Prediction

The past flourishing years of sequential location-based services began with the introduction of the Self-Attention Network (SAN), which quickly superseded CNN or RNN as the state-of-the-art backbone. Recent works utilize modified attention mechanisms or neural network layers to process spatial-temporal factors to realize fine-grained individual behavior pattern modeling. However, we argue these methods can be further improved due to the significant increase in the model's parameter scale or computational burden. In this paper, we first exploit two lightweight approaches, Rotary Time Aware Position Encoder (RoTAPE) and multi-head Interval Aware Attention Block (IAAB), to impel SAN by efficiently and effectively capturing spatial-temporal intervals among the user's visited locations, which require neither extra parameters nor a high computational cost. On the one hand, RoTAPE encodes the day- and hour-level timestamps into sequence representation simultaneously via a sinusoidal encoding matrix, and the corresponding time intervals can be explicitly captured by SAN. Specifically, the multi-level temporal differences are mutually independent to reflect the periodical pattern and jointly complete to measure the absolute time interval. On the other hand, IAAB, point- wise injecting the historical spatial-temporal intervals into the attention map, can promote SAN attaching importance to the spatial relations under the constraints of time conditions. Then, we design a novel MLP-based module, Spatial-Temporal Relation Memory (STR Memory), implemented with fully connected linear layers and matrix transpose operations. STR Memory, endowing the interactions inside historical intervals along different directions, can convert the historical intervals into spatial-temporal relations in future trajectories for accurate predictions. To this end, we propose an end-to-end mobility trajectory prediction framework, namely STiSAN
$^+$
, employing RoTAPE, stacking multiple layers of IAAB-based encoder-decoder architecture, and coupling with STR Memory. We conducted numerous experiments on six public LBSN datasets to evaluate our proposed algorithm. From Next Location Recommendation to Multi-location Future Trajectory Prediction, our STiSAN
$^+$
gains average 15.05% and 18.35% improvements against several state-of-the-art sequential models, respectively. Ablation studies demonstrate the effectiveness of RoTAPE, IAAB, and STR Memory under our framework. Moreover, we separately validate the extensibility and interpretability of RoTAPE and IAAB through non-sampled metric evaluation and visualization.