Capturing and Processing Sign Animations to a Portuguese Sign Language 3D Avatar

Usually, the deaf community faces enormous difficulties in communicating with the hearing community. Unfortunately, there are few systems available to the public to assist in the communication between the deaf community and the hearing community. Some countries, including Portugal, are trying to develop systems to mitigate this problem. Thus, in this paper, it is proposed a system that captures signs, using motion capture systems and produces the needed animations, represented by a 3D Avatar, so that a deaf user may understand them. This first implementation consists of a component, receiving text in Glosses format, from the Portuguese Sign Language, getting the corresponding animations from a database, and animating a 3D Avatar with these animations. The work being carried out is part of a larger system, that includes a user interface and another component that translates Portuguese Spoken Language to Portuguese Sign Language, through a Neural Network. The system aims to assist deaf people in communicating in all scenarios of everyday life. The first results show great potential for the deaf community to be able to use the proposed system.