Manifest: Public Speaking Training Using Virtual Reality

Public speaking is pivotal in personal and professional realms, evaluating communication skills. Employing VR training offers realistic practice, identifying weaknesses, improving speaking, and fostering confidence to overcome the fear of public speaking. This paper introduces a novel VR-based public speaking training system that comprehensively analyzes the speaker’s speech, body language, and nervousness levels. The system requires the use of a VR headset that immerses the user in a virtual classroom setting where they deliver speeches, while simultaneously recording the speech, capturing video through an external camera, and monitoring the user’s heart rate using a dedicated sensor. To evaluate the speech quality, the recorded speech is processed to calculate the scores of speaking rate, clarity, listenability, pronunciation, and number of pauses during the speech. The recorded video is further utilized to analyze body language, employing a pose estimation framework-YOLOv7. Furthermore, heart sensor data is employed to calculate the user’s level of nervousness. The system provides detailed analysis for each training session, providing users with performance feedback that can be accessed through the website. By integrating immersive VR technology, speech analysis, body language assessment, and heart rate monitoring, this novel public speaking training application provides a comprehensive way to evaluate public speaking skills and receive feedback and thus is a viable solution for coaching in the field of public speaking.