Merging Live and Static 360 Panoramas Inside a 3D Scene for Mixed Reality Remote Collaboration

We propose a new technique for Mixed Reality (MR) remote collaboration that uses live and static 360 panoramas inside a 3D scene. Prior work on MR remote collaboration demonstrated benefits of using a 360 panorama, or using a 3D scene. However, collaboration using 360 panorama offers limited spatial information. Our prototype system allows viewing and manipulating of 360 panoramas and visual cues inside a 3D scene, adding spatial information for enhanced contextual understanding between the remote collaborators. We designed a system that combines Augmented Reality (AR) and Virtual Reality (VR) to bridge the visual cues and information relative to the 3D space to help enhance the user experience and interaction in a larger scale MR remote collaboration. We describe our implementation in detail with a discussion of the design implications and potential benefits that could help improve MR remote collaboration.