From few to many: illumination cone models for face recognition under variable lighting and pose

We present a generative appearance-based method for recognizing human faces under variation in lighting and viewpoint. Our method exploits the fact that the set of images of an object in fixed pose, but under all possible illumination conditions, is a convex cone in the space of images. Using a small number of training images of each face taken with different lighting directions, the shape and albedo of the face can be reconstructed. In turn, this reconstruction serves as a generative model that can be used to render (or synthesize) images of the face under novel poses and illumination conditions. The pose space is then sampled and, for each pose, the corresponding illumination cone is approximated by a low-dimensional linear subspace whose basis vectors are estimated using the generative model. Our recognition algorithm assigns to a test image the identity of the closest approximated illumination cone. Test results show that the method performs almost without error, except on the most extreme lighting directions.

1 Introduction

It has been observed that “the variations between the images of the same face due to illumination and viewing direction are almost always larger than image variations due to change in face identity” [46]. As is evident in Figs. 1, 2, and 4, the same person with the same facial expression can appear strikingly different when light source direction and viewpoint vary. Fig. 1. Example images of a single individual in frontal pose from the Yale Face Database B showing the variability due to illumination. The images have been divided into four subsets according to the angle the light source direction makes with the camera axisSubset 1 (up to ), Subset 2 (up to ), Subset 3 (up to ), and Subset 4 (up to ). Fig. 2. Example images of a single individual, one from each of the nine different poses in the Yale Face Database B.