Automation Surprise Detection Using Facial Expression Recognition and Operation Input Values

In this paper, we propose a method to detect automation surprises caused by autonomous driving of electric wheelchairs using image processing. Automation surprises refer to the general confusion caused by discrepancies between the driver's and the system's perception of the driving situation, or by the driver's inability to fully grasp which mode the system is operatingin. Therefore, we propose a detection method of automation surprises based on facial expression recognition and the operational input values. By identifying sections where the driver's expression changes and excessive operational inputs to correct the vehicles unintended movement, we confirm that automation surprises can be detected from the racing game developed in Unity.