WTTFNet: A Weather-Time-Trajectory Fusion Network for Pedestrian Trajectory Prediction in Urban Complex

Pedestrian trajectory modelling in an urban complex is challenging because pedestrians can have many possible destinations, such as shops, escalators, and attractions. Moreover, weather and time-of-day may affect pedestrian behavior. In this paper, a new weather-time-trajectory fusion network (WTTFNet) is proposed to incorporate weather and time-of-day (WT) information to refine the predicted destination and trajectories. First, a word embedding is used to encode the WT information and its representation can be further optimized according to the loss function. Afterwards, a gate multimodal unit is used to fuse the WT information and preliminary pedestrian intent probabilities obtained from a preliminary baseline classifier. A joint loss function based on focal loss is used to co-optimize both the preliminary and final classifiers, which helps to improve the accuracy under possible class imbalances. Finally, a destination adapted trajectory model is used predict the trajectories guided by the predicted destination. Experimental results using the Osaka Asia and Pacific Trade Center (ATC) dataset shows improved performance of the proposed approach over state-of-the-art algorithms by 23.67% increase in classification accuracy, 9.16% and 7.07% reduction of average and final displacement error. The proposed approach may serve as an attractive approach for improving existing baseline trajectory prediction models when they are applied to scenarios with influences of weather-time conditions. It can be employed in numerous applications such as pedestrian facility engineering, public space development and technology-driven retail.