AWSIM-VR: A Tightly-Coupled Virtual Reality Extension for Human-in-the-Loop Pedestrian-Autonomous Vehicle Interaction

Effective communication between autonomous vehicles (AVs) and pedestrians is crucial for ensuring future urban traffic safety. While external Human-Machine Interfaces (eHMIs) have emerged as promising solutions, current evaluation methodologies — particularly Virtual Reality (VR)-based studies — typically rely on scripted or pre-defined autonomous vehicle behaviors, limiting realism and neglecting pedestrians’ active role in interactions. To address this, we introduce AWSIM-VR, a tightly-coupled VR extension of the AWSIM autonomous driving simulator, enabling real-time, human-in-the-loop pedestrian-AV interactions by directly integrating unmodified, real autonomous driving software (Autoware) into the simulation loop. Unlike previous systems, AWSIM-VR provides authentic, bidirectional interaction: pedestrians’ actions dynamically influence vehicle decision-making and eHMI responses in real-time, closely emulating real-world AV scenarios. In user studies directly comparing AWSIM-VR to existing methodologies, participants reported significantly higher perceived realism and immersion, underscoring the importance of authentic autonomous behaviors in VR-based pedestrian interaction research. By directly utilizing production-level autonomous driving stacks, AWSIM-VR represents a significant methodological advancement, enabling more realistic, effective, and safer development and evaluation of eHMIs and AV technologies.