Target-based evaluation of face recognition technology for video surveillance applications

This paper concerns the problem of real-time watch-list screening (WLS) using face recognition (FR) technology. The risk of flagging innocent travellers can be very high when deploying a FR system for WLS since: (i) faces captured in surveillance video vary considerably due to pose, expression, illumination, and camera inter-operability; (ii) reference images of targets in a watch-list are typically of limited quality or quantity; (iii) the performance of FR systems may vary significantly from one individual to another (according to socalled “biometric menagerie” phenomenon); (iv) the number of travellers drastically exceeds the number of target people in a watch-list; and finally and most critically, (v) due to the nature of optics, images of faces captured by video-surveillance cameras are focused and sharp only over a very short period of time if ever at all. Existing evaluation frameworks were originally developed for spatial face identification from still images, and do not allow one to properly examine the suitability of the FR technology for WLS with respect to the above listed risk factors intrinsically present in any video surveillance application. This paper introduces the target-based multi-level FR performance evaluation framework that is suitable for WLS. According to the framework, Level 0 (face detection analysis) deals with the system's ability to process low resolution faces. Level 1 (transaction-based analysis) deals with the ability to match faces in open-set problems, where target vs. non-target distributions are unbalanced. Level 2 (subject-based analysis) deals with robustness of the system to different types of target individuals. Finally, Level 3 (spatio-temporal analysis) allows one to examine the overall FR system discrimination by means of accumulating the recognition decision confidence over a face track, which can be used for developing more robust intelligent decision-making schemes including face triaging.The results from testing a commercial state-of-art COTS FR product on a public video data-set are shown to illustrate the benefits of this framework.