Keyword Search for Sign Language using Machine Translation

Sign language, the main mode of communication for the Deaf, is different from the written language in terms of both vocabulary and grammar. Lacking a proper written form, information retrieval from this visual language is a challenging task. Keyword search for sign language will benefit greatly from the use of written language text queries by (i) creating a middle ground for information retrieval, (ii) enabling the people outside of the Deaf community to learn sign language and (iii) making vast amount of parallel corpora between sign language and the written language available for research. In this work, we propose a cross-lingual keyword search paradigm for sign language based on neural machine translation, and analyze the effects of different decoding strategies on retrieval performance.