Encoding VR sessions: image-based techniques to record and inspect immersive experiences

Recording immersive VR sessions performed in virtual environments through HMDs - during public exhibits or permanent installations in a museum - may offer valuable insights on visitors propensities, physical workload, spatial affordances, focus and locomotion data. A first challenge raises from storage, where large amount of casual visitors need to be recorded on a daily basis, analyzed and possibly streamed to remote professionals. For each VR session in fact, size of collected dataset may grow very quickly. Another issue is related to the visual inspection of such data, providing a team of experts tools that allow to revisit the whole history of a single or multiple sessions. We propose, formalize and discuss image-based encoding techniques and quantized signals for compact and light "musical scores" that allow both common offline 2D image processing and interactive manipulation on modern desktop and mobile GPUs. We describe compact quantization methods and flexible volumetric observers, including location ignition and signatures. The paper discusses obtained results for different case studies where we employed such encodings, including desktop VR, WebVR and streaming of recorded sessions from cheap hardware.