The Effect of Visual and Auditory Modality Mismatching between Distraction and Warning on Pedestrian Street Crossing Behavior

Augmented reality (AR) headsets could provide useful information to users, but they may also be a source of distraction. Previous works have explored using AR to enhance pedestrian safety by providing real-time warnings, but there has been little research on the impact of modality matching between distractions and warnings on pedestrian street crossing behaviour. We conducted a VR experiment using a within-subjects 2-by-2 design (N=24) with four conditions: (auditory distraction, visual distraction) $\times$ (auditory warning, visual warning). When experienced conditions with mismatched modalities, participants exhibited more cautious street crossing behaviours, such as reduced walking speed, and increased scan range after receiving the warning, and significantly faster reaction times to the incoming vehicle. The participants also expressed a preference for warnings to be presented in a modality different from the distraction. Our findings suggest that in the context of utilizing AR for pedestrian road safety, future AR interfaces should incorporate a warning modality that differs from the one causing distraction.