Architecture Design and Development of an On-board Stereo Vision System for Cooperative Automated Vehicles

In a cooperative automated driving scenario like platooning, the ego vehicle needs reliable and accurate perception capabilities to autonomously follow the lead vehicle. This paper presents the architecture design and development of an on-board stereo vision system for cooperative automated vehicles. The input to the proposed system is stereo image pairs. It uses three deep neural networks to detect and classify objects, lane markings, and free space boundary simultaneously in front of the ego vehicle. The rectified left and right image frames of the stereo camera are used to compute a disparity map to estimate the detected object's depth and radial distance. It also estimates the object's relative velocity, azimuth, and elevation angle with respect to the ego vehicle. It sends the perceived information to the vehicle control system and displays the perceived information in a meaningful way on the human-machine interface. The system runs on both PC (x86_64 architecture) with Nvidia GPU, and the Nvidia Drive PX 2 (aarch64 architecture) automotive-grade compute platform. It is deployed and evaluated on Renault Twizy cooperative automated driving research platform. The presented results show that the stereo vision system works in real-time and is useful for cooperative automated vehicles.