Kannada-English Code-Mixed Speech Synthesis

Code-mixing is the situation where words and phrases from two or more different languages are used inter-changeably within one sentence or utterance. With the increasing number of bilinguals and multilinguals in today's world, there is heavy evidences of code-mixing in many scenarios. However their presence affects the accuracy of current natural language processing and speech processing systems, since currently available speech-to-text and text-to-speech systems are not viable to translate speech which consists of two or more languages mixed together to a target language, as they assume the text and speech to all come from a single language. In the case of Dravidian languages, especially Kannada-English code-mixing, there are very few datasets available, and lesser so which consist of speech data. In order to get closer to rectifying this problem, we present a generative adversarial architecture to synthesize code-mixed speech in Kannada-English using monolingual utterances in Kannada. We are able to generate sentences with a good FAD (Frechet Audio Distance) score of around 14.490, using a very small training dataset.