Multimedia Databases for Video Indexing: Toward Automatic Face Image Registration

Pose-invariant face recognition systems for multimedia indexing require the prior registration of face images at multiple poses in a database, but it can be problematic and laborious to obtain and register appropriate imagery. We aim to automate the process by constructing 3D face models from the imagery available for registration, and then using the constructed models to generate templates for the face recognition system. The first step in the model construction process is the estimation of the generalized pose (scale, position, 3D orientation relative to the camera) of the face in each frame of the registration imagery, and the 3D positions of a number of feature points on the face. This is followed by warping the 3D model to fit the estimated 3D feature points, and mapping facial texture from the registration imagery onto the model. In this paper we outline (i) an algorithm for estimating the generalized pose and shape (3D feature point locations) from 2D feature point tracking data, and (ii) a texture mapping algorithm that combines texture regions from all of the available imagery. We show experimental results and discuss issues that remain in applying the method in practice as part of a multimedia indexing system.