Mugshot database acquisition in video surveillance networks using incremental auto-clustering quality measures

Face recognition has primarily focused on recognizing and matching face images against large, controlled databases of frontal views. Many of these techniques perform well against databases that have been collected from a reduced set of viewpoints, under controlled lighting, and are normalized for scale. Acquisition of these databases, however, particularly in unconstrained environments, remains a challenge. We present a real-time technique to acquire a mugshot database automatically from a video surveillance network. Mugshot extraction is a twofold problem. First, faces are detected and tracked in all cameras of the network. Face targets are analyzed to determine which frames represent actual mugshots capable of supporting subsequent matching and recognition. Next, mugshot candidates are evaluated based on their ability to improve the quality of the incrementally constructed database. We introduce a database quality measure, which assigns high value to mugshots of previously unseen subjects or mugshots that do not decrease separability of existing clusters. The quality measure is discounted for mugshots that are redundant or increase the intra-cluster spread. Results demonstrate that automatic acquisition of a high-quality database from a twelve-camera network is feasible. The quality of these databases is demonstrated using traditional methods to match faces accurately against the acquired database.