Improvement of Autonomous Vehicles Trust Through Synesthetic-Based Multimodal Interaction

Trust is the key factor for people to accept autonomous vehicles(AVs). Existing studies have reported that multimodal interaction would enhance people's trust in AVs. However, these researches mainly focus on the superposition effect between sensory channels, and lack on the research of correlation between different sensory channels and its influence on AVs trust. Therefore, we innovatively introduce synesthesia theory for the research of improving AVs trust. We present an AVs multimodal interaction model based on audio-visual synesthesia theory, and finally prove that the model has a definite effect on improving AVs trust by experiments. Firstly, 82 participants are recruited and assigned into two groups: Group A (non-synesthesia group) and Group B (synesthesia group). They conduct an experimental driving experienced normal traffic conditions (NTC) (turning, traffic lights, over and limit speed prompts) and emergency traffic condition (ETC) (sudden braking of the car in front, temporary lane change, pedestrian thrusting) while completing a secondary task. Then, we conduct a survey (questionnaire and interviews) to evaluate the attitude about trust, technical competence, situation management and perceived ease of use after participants finished experimental driving. The results demonstrate that synesthetic-based multimodal interaction (SBMI) can more effectively remind people of relevant information especially under ETC. SBMI model is more effective than single information stimulus or non-synesthetic audio-visual information stimulus not only in terms of information transmission efficiency and effect, but also in terms of output response/ action. The results also show that SBMI contributes to the improvement of AVs trust. These findings provide evidence on the importance of SBMI to the improvement of AVs trust. The findings of this study will be helpful to the future design of AVs interaction system.