A Survey on Deep-Learning Methods for Pedestrian Behavior Prediction from the Egocentric View

This paper surveys deep-learning-based pedestrian behavior prediction algorithms from the ego-vehicle's perspective. To safely deploy autonomous vehicles, pedestrian behavior must be understood and predicted for safe and efficient vehicle-pedestrian interactions. With more and more algorithms proposed in the past 2–3 years, it is vital to summarize how the state-of-the-art algorithms estimate pedestrian behavior. To achieve a full view of current algorithms in this domain, in this paper we review (1) prediction output types, (2) network input features, (3) network architecture, and (4) the datasets available for training/testing. As pedestrian behavioral studies show many factors impact a pedestrian's willingness to cross the street, prediction algorithms are evolving to include more rich visual annotations accordingly. Networks architecture is changing from temporal to spatio-temporal networks to account for the influences traffic agents have on one another as well. More innovative benchmark datasets are also published to support more research efforts. The survey depicts the current research frontier in predicting pedestrian behaviors to build the foundation for future research in the area.