Learning multi-lane trajectories using vehicle-based vision

Safe operation of a motor vehicle requires awareness of the current traffic situation as well as the ability to predict future maneuvers. In order to provide an intelligent vehicle the ability to make predictions, this work proposes a framework for understanding the driving situation based on vehicle mounted vision sensors. Vehicles are tracked using Kalman filtering based on a vision-based system that detects and tracks using a combination of monocular and stereo-vision. The vehicles' full trajectories are recorded, and a data-driven learning framework has been applied to automatically learn surround behaviors. By learning based on observations, the ADAS system is being trained by experience. Learned trajectories have been compared between dense and free-flowing traffic conditions. Preliminary experimental results using real-world multi-lane highways show the basic promise of this approach. Future research directions are discussed.