Video communication system using portrait animation

Novel video and speech communication schemes are investigated for implementing a chat-style communications system that can be applied to mobile communications environments characterized by limited terminal performance and transmission throughputs. A method is developed of automatically generating simplified portraits that capture the essence of a person's unique facial features from photos taken with a video camera. Based on these simplified images, an animated portrait communications system is then proposed that conveys mouth movements and a range of facial expressions by animation. This only requires transmitted data representing movements of the mouth and to select the appropriate expression, and thus can support video communications at a much lower bit rate than if actual photos are sent back and forth. Next we propose a catch-mode store-and-forward communication method for speech that momentarily stores a speaker's words until one sentence or statement is finished, then transmits the speech. This approach is suitable for implementation on even relatively low-performance data terminals such as PDAs since speech is not input and output simultaneously. It also has the virtue of permitting users to input speech anytime they want at will, since it does not involve any complicated procedure of requiring users to acquire a token to be given a chance to speak. Finally a prototype Digicom Club terminal was constructed that supports chat-style video and speech communication based on the proposed animated portrait and catch-mode voice communications systems.