Virtual characters with affective facial behavior

This paper describes an application that generates a simulation of a jury with one to three virtual humans capable of exhibiting facial and body expressions controllable in real-time. This control is performed through an interface that combines facial action elements (AU-Action Units, as described in the Facial Action Coding System), with upper-body postures. The level of detail of this control offers a range of possible combinations to obtain emotional expressions. Besides offering the possibility of controlling the postures of the virtual characters, the application allows the user to choose, among a pre-defined set, the virtual characters that compose the jury, and is able to introduce in the simulation some extra, potentially distractive or annoying events. We envisaged two contexts for using this application: i) assisting psychotherapists in exposure therapy of patients suffering from anxiety of public speaking, particularly in front of a jury in an assessment situation; ii) supporting nonverbal behaviour research carried out by psychologists. The development of the application has been closely monitored by a psychologist that is part of our team. This application is a low-cost approach, which uses only free software and models and resorts to common equipment; it is easy to install and use by people without expertise in informatics. So far, we have performed an evaluation with therapists in the first application context, obtaining encouraging results.