A systems engineering approach to micro expression facial motion capture with structured light

A team of students at Southern Methodist University (SMU) is working with the U.S. Department of Defense (USDOD) to improve the U.S. Marine Corps training program, specifically training simulations in the U.S. to help soldiers understand non-verbal communication clues such as facial ticks. Without using facial markers an actor's face is captured and in real time transferred onto a 3-D avatar. Using one scientific camera and a process called structured light, the user's image is captured using a sinusoidal wave pattern projected over the user's face which creates a depth map of the face similar to a topographical map of the user's face. an edge detection algorithm is then applied to the depth map capturing the desired facial features, creating a wire frame. This wire frame rendering of the user's face is translated onto a mesh, creating a realistic looking avatar. Then the image is rendered and displayed to the soldier.