Multi-Sensor Fusion Perception System in Train

Environment perception is one of the most crucial modules in a self-driving system. In an open subway environment, pedestrians, equipment boxes and other unknown obstacles often appear. The perception module is required to quickly and accurately recognize the obstacle and measures the corresponding distance data. This paper proposes a detection and ranging fusion method based on one Lidar and two different focal length cameras, which be applied in the subway system, to alert drivers to possible obstacles and assist brake. First, the relative transformation between Lidar and camera is calibrated to find the intrinsic and extrinsic matrices. Modified SSD network is trained on the self-built dataset to detect the potential obstacle and produces the final detection bounding box. Then, rail track segmentation network RailNet is applied to obtain rail shape features, which offers a region of interest (ROI) for sensing systems in point cloud clustering algorithms. Further, the fusion information is utilized to estimate the distance of objects detected by the SSD detector. The proposed method can realize real-time pedestrian detection and range on the onboard embedded platform Jetson Xavier, which satisfies the subway environment's perception requirements.