By Topic

A real-time interactive nonverbal communication system through semantic feature extraction as an interlingua

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Jin Hou ; Graduate Sch. of Eng., Hokkaido Univ., Sapporo, Japan ; Y. Aoki

There has been a growing interest in the use of networked virtual environment (NVE) technology to implement telepresence that allows participants to interact with each other in shared cyberspace. In addition, nonverbal language has attracted increased attention because of its association with more natural human communication, and especially sign languages play an important role for the hearing impaired. This paper proposes a novel real-time nonverbal communication system by introducing an artificial intelligence method into the NVE. We extract semantic information as an interlingua from the input text through natural language processing, and then transmit this semantic feature extraction (SFE) to the three-dimensional (3-D) articulated humanoid models prepared for each client in remote locations. Once the SFE is received, the virtual human is animated by the synthesized SFE. Experiments with Japanese and Chinese sign languages show this system makes the real-time animation of avatars available for the participants when chatting with each other. The communication is more natural since it is not just based on text or predefined gesture icons. This proposed system is suitable for sign language distance training as well.

Published in:

IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans  (Volume:34 ,  Issue: 1 )