Motion history for facial action detection in video

Enabling computer systems to recognize human facial expressions is a challenging research problem with many applications in behavioral science, medicine, security, and human-machine interaction. Instead of being another approach to automatic detection of prototypic facial expressions of emotion, this work attempts to analyze subtle changes in facial behavior by recognizing facial action units (AU, i.e. atomic facial signals) that produce expressions. This work proposes AU recognition based upon multilevel motion history images (MMHI), which can be seen as an extension to temporal templates introduced by Bobick and Davis. By recording motion history at multiple time Intervals (i.e., multilevel MHI) instead of recording it once for the entire image sequence, we overcome the problem of self-occlusion which is inherent to temporal templates original definition. For automatic classification of an input MMHI-represented face video in terms of 21 AU classes, two approaches are compared: a sparse network of Winnows (SNoW) and a standard kNearest neighbour (kNN) classifier. The system was tested on two different databases, the MMI-Face-DB developed by the authors and the Cohn-Kanade face database.