Temporal Attention With Gated Residual Network for Stepwise Multimodal Trajectory Prediction

This article presents a temporal attention with gated residual network (TAGRN) for multimodal trajectory prediction tasks. First, a temporal attention encoder based on a temporal attention is used to identify the time points in the observation sequence that have a greater impact on future trajectories. The gated residual network (GRN) is applied to focus on and encode the crucial temporal features of the input sequence. The GRN assigns higher weights to more important multimodal input features. Next, a stepwise autoregressive (SAR) decoder is employed to gradually predict agents’ future multimodal trajectories. This decoder estimates multiple key feature points of the agents at the next time step using a Gaussian distribution, a gated recurrent unit, and a feedforward neural network. An autoregressive strategy is then applied to iteratively infer the multimodal distribution of long-term future trajectories. The proposed model is validated and quantitatively evaluated on three first-person view datasets—pedestrian intention estimation (PIE), trajectory inference using targeted action priors network (TITAN), and the newly introduced Taiwan Risky Road Dataset (TaRDD)—as well as one third-person view dataset [stanford drone dataset (SDD)]. The results demonstrate that the model achieves the state-of-the-art performance across all datasets.