A Visualization Method of Facial Expression Deformation Based on Pore-Scale Facial Feature Matching

Facial expression is the main research content of acquiring human psychology and developing human-computer interaction. The expression is made up of deformation and movement of each facial muscle, especially the un-posed micro expression is difficult to study because of the small degree of deformation. How to find an intuitive and interpretable method to visualize the deformation of facial expressions is an important and neglected task. In this paper, we present a new method of facial expression deformation visualization based on pore-scale facial feature matching. We use PSIFT (Pore Scale-Invariant Feature Transform) to extract a large number of facial pore-scale keypoints. Then GMS (Grid-based Motion Statistics) and VFC (Vector Field Consensus) are used to get robust matching pairs of facial feature points, so that to obtain multi-frame pore-scale keypoints tracking trajectories with time. Based on Munsell Color System, we transform matching groups into displacement vector field, Gaussian filter kernel and grid-based principal direction vector are computed to visualize the amplitude and direction of facial muscle movement. Compared with other methods of displacement field visualization (such as Horn-Schunck Optical Flow, SIFT Flow and Line Integral Convolution), our method achieves the best visualization effect in the field of facial expressions deformation on BP4D database and the collected high resolution facial expression video, the research on this subject conducive to the further development of others, such as facial expression recognition, expression modeling, expression generation and so on.