3D immersive karaoke for the learning of foreign language pronunciation

Foreign language pronunciation learning relies on listening, discriminating, identifying, repeating and, finally, pronouncing. Computer assisted pronunciation training requires strategies that capture the attention of the learners and guide them throughout the learning pathway. In this paper, we introduce an immersive scenario for creating appropriate learning conditions. In the creation of this scenario, we explore the usage of virtual reality and 3D audio. The proposed learning interaction is orchestrated by a spoken karaoke. We present our approach for the processing of spoken and visual interaction modalities and discuss how technological challenges are addressed in order to enable the learner's self evaluation. Moreover, we propose an automatic pronunciation assessment based on both audio and visual modalities. The evaluation feedback is returned to the learner using 2D and 3D visualization techniques.