Towards a Personalized Multimodal System for Natural Voice Reproduction

Loss of voice, which may be due to problems in various organs of the human voice system is a major disability which most of the time results in social isolation. In most cases of complete voice loss, the problem cannot be improved over time and despite the fact that methods have been developed to reconstruct voice, none come close to the natural pitch of the person's voice. The present work proposes a novel personalized system that integrates audiovisual information and autonomous learning in order to reconstruct a disabled person's voice. The system makes use of a smartphone's camera and microphone while the majority of computations will not be performed locally but in the cloud. Using computational methods for signal processing as well as machine learning algorithms we believe that, a voice similar to the user's own voice can be reproduced. Finally the aforementioned technologies will be incorporated into a personalized multimedia interface and and the system will be evaluated during a multi-phase period by patients with serious chronic voice problems.