Towards Dual-Modal Crowd Density Forecasting in Transportation Building

Crowd density forecasting in transportation building has valuable applications involving security, crowd management, and service design. The existing methods lack the prediction performance to forecast long-term (minutes-long) crowd density, which specializes in being sensitive to the external condition. Thus, we propose a method that can combine dual-modal information: the surveillance video streams and the transportation schedule information to forecast the future crowd density in the transportation building. The model utilizes the temporal convolution layers to extract the time dependence of the video streams and the transportation schedule. The pooling with an assignment matrix technique is used to learn the correlation between the video and the transportation schedule information. The predictor fuses both information and uses the Gated Recurrent Unit (GRU) layers to predict the crowd density. The experimental results show that our method could effectively benefit from the dual-modal information and give more accurate prediction results.