A Software QA Framework for Autonomous Vehicle Open Source Application: OpenPilot

As the deployment of autonomous vehicles (AV) grows, ensuring their reliability and safety becomes paramount. The need for rigorous testing and validation methods is surging. This paper focuses on investigating the resilience of OpenPilot, an open-source driving agent for assisted driving systems, against faults and environmental conditions affecting sensor data, targeting faults that directly impact machine learning (ML) and perception systems, known to be a significant cause of disengagement incidents in AV. To assess the effectiveness and coverage of Open-Pilot's functionalities, we employ standard model-driven test engineering methodology graph coverage testing. A systematic and comprehensive modeling of OpenPilot using graph coverage methodology is proposed, covering three programming scripts and yielding 16 major test case scenarios. This approach enables graph coverage testing on OpenPilot, facilitating evaluation of its performance, robustness, and safety measures. By systematically exploring the system's functionalities and scenarios, we ensure OpenPilot performs as expected and effectively mitigates safety hazards, contributing to the enhancement of autonomous vehicle safety and building confidence in autonomous driving technology.