Dealing with unknowns in machine translation

An 'unknown' is defined as a word for which there is no entry in the dictionary used by the translation system. In general, a text may contain several unknowns. These words may be names, acronyms, abbreviations, terminology or foreign words. It is a common practice in India to mix English words in Hindi and other Indian languages, and vice-versa. However, the grammatical rules in the construction of gender, number, verb nominalization or forms conform to those for the language used, irrespective of their origin. This gives rise to frequent encounters with unknown words in day-to-day communications. A machine translation system has to provide a mechanism for handling such unknowns. Spelling mistakes are yet another source that contributes to these unknowns. In this paper, we describe the strategy being adopted in our system for machine-aided translation from English to Hindi. No attempt has been made to expand the vocabulary by deriving the meaning of the unknown words. Instead, once an unknown is identified, a transliteration in Hindi with appropriate suffixes or appendages is used to substitute for their meaning. We use predictive parsing and a number of heuristics to identify the type of unknown.