Exploring the Design Space of an Augmented Display for Conveying Facial Expressions for People with Autism

Facial expression is considered as the most intuitive and effective way of conveying one's emotion among other nonverbal interactions.However, people with autism have limited access to this rich communication channel due to their inability to read facial expressions. To help them be aware of others' emotions, we developed a CNN-based facial expression recognition system using Microsoft Hololens and explored three different modes for displaying facial expressions of a conversation partner varying the levels of explicitness. Subjective feedback from a preliminary study with 6 pilot participants suggests that each mode is worth investigating for serving people with various needs and preferences who wish to receive augmented visual hints on others' emotion.