An embedded multilingual speech recognition system for Mandarin, Cantonese, and English

Here, we propose a small-footprint speaker-independent, multilingual system for isolated word recognition of Mandarin, Cantonese, and English. The baseline system got very promising results without any phoneme shared between different languages. By sharing phonemes, the memory and computational complexity was reduced by about 40%. Nonnative, accented speech recognition and mixed language words support are the distinguishing features of our system. Automatic language identification (LID) is one of the key elements in language-independent automatic speech recognition (ASR) systems. LID perfomance is also analyzed in addition to the engine performance of the proposed system. Supervised Bayesian online adaptation was proved to be effective in compensation for accent mismatch, environment mismatch, as well as for modeling inaccuracy introduced by combined training.