An Integrated Vision-Based Perception and Control for Lane Keeping of Autonomous Vehicles

This paper proposes a vision-based control method for autonomous vehicle lane keeping. This paper aims to provide a reliable alternative solution for localization and path planning-related challenges in environments where GPS is unavailable or unreliable, such as rural areas with dense forest cover, where tree canopies may obstruct GPS signals, and urban landscapes characterized by tall buildings that can similarly disrupt navigation accuracy. The proposed method consists of a robust lane detection algorithm to generate a reference path of the vehicle and a model predictive controller (MPC) for tracking the reference path. The lane detector extracts lane markings from image frames to determine ego-lane boundaries from which the lane center is calculated in the vehicle’s coordinate frame. The MPC uses a kinematic vehicle model to generate the lateral and longitudinal control values necessary for smoothly tracking the reference path. The proposed technique has been implemented and tested on a Lincoln MKZ hybrid vehicle equipped with a computing platform having Intel’s quad-core Xeon processors. The developed framework was experimentally validated by deployment in a rural test track. Experimental results show that the proposed vision-based lane detection method performs well under various challenging road conditions such as shadows, road texture variations, interference from other road signs, and missing lane boundaries. The multi-threaded implementation of lane detection and the MPC allowed us to run the integrated system at the speed of 25 HZ. The integration of lane detection and MPC resulted in smoothly keeping the car in the center of the lane over a curved test track while respecting the physical constraints of the car.