CNN-based 360° Scene Recognition for Automatic Generation of Omnidirectional Scent Effects

Multiple approaches aim to enhance user experience in the delivery of immersive video content. The popularisation of VR, combined with recent advances in mulsemedia technology has improved access to immersive visual and olfactory stimuli. Synchronising multiple scent dispensers positioned around the user when watching 360° videos can more accurately indicate the location of scent sources, guiding users to move their heads accordingly to the indicated directions. However, the manual annotation process required to add mulsemedia effects is labour-intensive, limiting the availability of content with sensory enhancements, particularly when using multiple scent dispensers from various directions. Addressing this issue, this paper introduces OmniScent-CNN, an innovative solution to automate the diffusion of scents from different directions in a VR environment using Convolutional Neural Networks (CNNs) for scene recognition. Multiple instances of the solution were tested, employing a number of CNN architectures. The results demonstrated that olfaction accuracy can reach up to 71.28% with the ResNet-18 model. Furthermore, user perceptual tests revealed excellent results, with 87.5% of participants agreeing or strongly agreeing that the scents enhanced their enjoyment of the experience. This indicates the feasibility of automating the process of synchronising omnidirectional scents based on 360° scene recognition.