Do You XaRaoke? Immersive Realistic Singing Experience with Embodied Singer

We have developed an immersive karaoke experience that allows users to sing their favorite songs in front of a simulated audience. The karaoke experience is designed within an immersive stadium scene with a simulated audience and stage lights that synchronize with the song's beats and displayed lyrics. Unlike VR-based karaoke commercial solutions, users can even see their real bodies as video-based self-avatars through the use of a deep learning network and a real microphone without using VR controllers. Preliminary results from a subset of 17 participants validate the developed prototype and provide insights for future improvements.