Beyond Realities: Designing Cross-Reality Interactions in the Immersive Narrative Experience

As closed head-mounted displays (HMDs) limit users’ reality awareness, the cross-reality (CR) interaction has emerged to introduce realities across different stages on the Reality-Virtuality Continuum (RVC). Although current CR interaction presents passthrough to switch different environments, it results in visual differences that limit a truly immersive and coherent experience. Therefore, it is essential to design seamless and bidirectional CR interactions, rather than sudden switching. In this paper, we propose four CR interactions to bind the virtual and real elements contextually in the highly immersive experience. We designed an immersive narrative "Beyond Realities", and implemented four CR interactions in it: Portal Passthrough, Virtual Guidance, Object Passthrough, and Bystander Avatar. We investigated further through a qualitative study (n=11), to explore the impacts of CR interactions on immersive experience, narrative comprehension, and interaction usability. In the highly immersive experience, CR interactions can offer a harmonious blend of virtual and real scenarios.