Digital facial augmentation for interactive entertainment

Digital projection technology allows for effective and entertaining spatial augmented reality applications. Leveraging the capabilities of reasonably accurate object tracking using commodity cameras and/or depth sensors to determine the 3D position and pose of objects in real time, it is possible to project dynamic graphical content on arbitrary surfaces, such as a person's face. Coupling these capabilities with a simple drawing application, participants can have the experience of “painting” on someone's face, or even on their own, by observing the projection in a mirror. Similarly, integrating 2D rigid-body, fluid and gravity simulation, one may interact with virtual objects projected on their own face or body.