Safety Implications of Variability in Autonomous Driving Assist Alerting

Advanced Driving Assist Systems (ADAS) are on the rise in new cars, including versions that embed artificial intelligence in computer vision systems that leverage deep learning algorithms. Because these systems, at the present time, cannot operate in all operational driving domains, they employ some type of driver monitoring system for assessing driver attention, so that drivers can effectively take control if and when an ADAS system can no longer control the car. To determine the reliability of a driver alerting system when linked to autonomy that leverages deep learning, a set of increasingly complex tests were conducted on three Tesla Model 3 vehicles. Tests were conducted on a highway and a closed test track to test road departure and construction zone detection capabilities. Results revealed significant between- and within-vehicle variation on a number of metrics related to driver monitoring, alerting, and safe operation of the underlying autonomy. In some cases, cars performed better than expected but all cars exhibited both inconsistent and unsafe behaviors as well as poor driver alerting. These results highlight that a post-deployment regulatory process is ill-equipped to flag significant issues in vehicles with embedded artificial intelligence.