The AESOP continuous speech understanding system

The continuous speech understanding system AESOP that we are presently developing at LIMSI, has gone through several phases since the first preliminary version, AESOP0 in 1978. This version allowed for man-machine speech communication in real time, with unrestricted text-to-speech synthesis. About 80% of the sentences were correctly recognised, for a language having an average branching factor of 10. In the AESOP 1 system, sentence recognition is 90 %,(95% on words) for the same language. Apart from this improvement, the system has been designed to be used easily for artificial languages, by adding graceful interaction to define semantic grammar (including automatic grapheme-to-phoneme translation of the vocabulary) and talker adaptation. Simultaneously, we developed a method for diphone recognition of continuous speech, using a 1089-diphone dictionary. Although the results are presently mediocre on pure phoneme recognition, its use as the acoustic level of the SUS gave good results. All those systems use top-down, left-to-right strategy, with beam-search. A bottom-up approach has also been tested for future use, having a tree-structure vocabulary; this still yields the same results.