A Hybrid Lightweight Model for Real-Time Pedestrian Perception and Motion Forecasting in Autonomous Driving

Pedestrian safety is an essential issue for autonomous vehicle (AV) systems in complex and dynamic urban scenes. However, the current deep learning-based pedestrian detection and trajectory prediction methods usually meet the bottleneck that the trade-off between accuracy and inference time makes them difficult for implanted in real-time and resource-constrained edge devices. To mitigate such limitations, this paper develops HybridNet-YT, a hybrid lightweight model simultaneously incorporating both person detection and motion prediction in one efficient network. It has an EffcientConv backbone designed for fast and efficient feature extractions and a dual-head structure: a YOLO-like anchor-free detection head and a Transformer based forecasting head for real-time pedestrian trajectory prediction. Furthermore, an attention fusion module is utilized for more adequate spatial-temporal context integration. This is evidenced by experiments on JAAD and PIE datasets where HybridNet-YT achieves superior detection performance (mAP), low trajectory prediction errors and real-time speeds (< 15 ms latency), allowing it to be readily deployed on the edge. HybridNet-YT provides a practical solution for generating pedestrian proposal to promote safety and response in real-world driving for AVs.