SISAd: Advertisement Implantation in VR Video with Saliency-constrained Instance Segmentation

This paper investigates the issue of advertisement implantation in virtual reality (VR) video for potential consumer electronics applications. Unlike traditional video advertisement implantation, VR video advertisement implantation faces new challenges as it needs to consider both user immersion and user attention. In this paper, we propose a deep neural network called SISAd to identify candidate spaces for advertisement implantation in VR video scenes. Particularly, we combine instance segmentation and saliency detection techniques to automatically detect and locate regions that attract users’ visual attention, enabling natural and engaging advertisement implantation. To train our SISAd network, we construct a VR video advertisement implantation dataset (VRAD). This dataset can also be reused by future relevant research. We evaluate our approach against public benchmarks and show that it outperforms other alternative solutions for advertisement implantation. Through this research, we provide new insights and solutions for VR video advertisement implantation for future consumer electronics applications.