Immersive Experience Prototyping: Using Mixed Reality to Integrate Real Devices in Virtual Simulated Contexts to Prototype Experiences with Mobile Apps

Interaction and augmentation strongly depend on the context of use and the respective human aspects. In this paper, we present the use of Mixed Reality (MR) for the creation of experience prototypes and the evaluation of mobile applications in a (simulated) usage context. The novelty is the integration of real mobile devices into MR according to the augmented virtuality principle and the mirroring of the screen from the real device to the virtual replica. This preserves the user experience and the full range of functions when using the mobile device in MR allowing it to be operated as in reality. In the MR environment it is possible to simulate usage contexts and to investigate the user experience with mobile applications. First, we show the technical concept and then we demonstrate the approach with a prototype smartphone app for a mobility assistant in two scenarios.