Vision-based recognition of pedestrian crossing intention in an urban environment

Autonomous moving platforms (like self-driving cars and mobile robots) need to understand the behaviors of pedestrians for better performance. Recognizing pedestrians' crossing intention is one of the most critical capabilities of self-driving cars to guarantee safe operations in an urban environment. Some researchers utilize pedestrians' trajectories to identify their crossing intention, whereas ignoring the informative appearance. This work studies two vision cues which reveal pedestrians' crossing intention. A looking/not looking classifier is built to capture eye contacts between pedestrians and self-driving cars. Such eye contacts are considered to be highly related to pedestrians' crossing intention. A crossing/not crossing classifier is designed to catch early actions before pedestrians starting to cross, like limb movements. Data augmentation is used before training both classifiers due to limited training samples. Meanwhile, both classifiers are optimized with focal loss, which can relieve the issue of easy/hard samples. Experimental evaluations verify the effectiveness of the proposed vision cues in recognizing pedestrians' crossing intention. Comparisons with the baseline indicate the superiorities of the proposed strategies in training the two classifiers.