The global dimensionality of face space

A low-dimensional representation of sensory signals is the key to solving many of the computational problems encountered in high-level vision. Principal component analysis (PCA) has been used in the past to derive such compact representations for the object class of human faces. Here, with an interpretation of PCA as a probabilistic model, we employ two objective criteria to study its generalization properties in the context of large frontal-pose face databases. We find that the eigenfaces, the eigenspectrum, and the generalization depend strongly on the ensemble composition and size, with statistics for populations as large as 5500, still not stationary. Further, the assumption of mirror symmetry of the ensemble improves the quality of the results substantially in the low-statistics regime, and is also essential in the high-statistics regime. We employ a perceptual criterion and argue that, even with large statistics, the dimensionality of the PCA subspace necessary for adequate representation of the identity information in relatively tightly cropped faces is in the 400-700 range, and we show that a dimensionality of 200 is inadequate. Finally, we discuss some of the shortcomings of PCA and suggest possible solutions.