Pedestrian Trajectory Prediction in Heterogeneous Traffic Using Pose Keypoints-Based Convolutional Encoder-Decoder Network

Future pedestrian trajectory prediction offers great prospects for many practical applications. Most existing methods focus on social interaction among pedestrians but ignore the factors that heterogeneous traffic objects (cars, dogs, bicycles, motorcycles, etc.) have significant influence on the future trajectory of a subject pedestrian. Also, the walking direction intention of a pedestrian may be referred by his/her pose keypoints. Considering this, this work proposes to predict a pedestrian’s future trajectory by jointly using neighboring heterogeneous traffic information and his/her pose keypoints. To fulfill this, an end-to-end pose keypoints-based convolutional encoder-decoder network (PK-CEN) is designed, in which the heterogeneous traffic and pose keypoints are modeled as input. After training, PK-CEN is evaluated on manifold crowded video sequences collected from the public dataset MOT16, MOT17 and MOT20. Experimental results demonstrate that it outperforms state-of-the-art approaches, in terms of prediction errors.