A Novel Framework for Hand Visualization in Web-Based Collaborative XR

Many extended reality (XR) applications are platform-specific, making accessibility and cross-platform collaboration difficult. Web-based collaborative XR can enhance adoption of XR technologies, using the browser as a platform-independent interface. However, challenges arise from the browser environment, such as performance limitations. To this end, we present a WebXR-based framework for hand interaction in cross-platform collaboration in XR. A network structure and methods for collaborative and individual object manipulation complement the integrated hand tracking. Three different fidelity levels to represent the hands of remote users were implemented to accommodate different performance capabilities. Concepts ranged from virtual hands to discrete poses with abstract objects. A sample application was implemented with a puzzle task. Two users collaborated in the browsers of the Microsoft HoloLens 2 and the Meta Quest 2. Qualitative and quantitative data on user performance (n=9), and frame rate recordings (n=1) were collected. All users were able to solve the puzzle together quickly and intu-itively. The Quest environment was preferred, as there were more performance issues with the HoloLens. Hand interaction was well-received and proved to be sufficient as the only form of communication. Simpler representations of the hands lead to a higher frame rate, whereby the effects were device-dependent. The impact on task performance was low. Hand interaction enables an intuitive exchange of objects and basic communication in cross-platform collaboration via browsers. Depending on the XR environment, however, device-specific performance limitations must be taken into account by modulating the amount of data and rendering effort.