Surveillance System for Real-Time High-Precision Recognition of Criminal Faces From Wild Videos

As violent criminals, such as child sex offenders, tend to have high recidivism rates in modern society, there is a need to prevent such offenders from approaching socially disadvantaged and crime-prone areas, such as schools or childcare centers. Accordingly, national governments and related institutions have installed surveillance cameras and provided additional personnel to manage and monitor them via video surveillance equipment. However, naked-eye monitoring by guards and manual image processing cannot properly evaluate the video captured by surveillance cameras. To address the various problems of conventional systems that simply store and retrieve image data, a system is needed that can actively classify captured images in real-time, in addition to assisting surveillance personnel. Therefore, this paper proposes a video surveillance system based on a composable deep face recognition method. The proposed system detects the faces of criminals in real time from videos captured by a surveillance camera and notifies relevant institutions of the appearance of criminals. For real-time face detection, a down-sampled image forked from the original is used to localize unspecified faces. To improve accuracy and confidence in the recognition task, a scoring method based on face tracking is proposed. The final score combines the recognition confidence and the standard score to determine the embedding distance from the criminal face embedding data. The blind spots of surveillance personnel can be effectively addressed through early detection of criminals approaching crime-prone areas. The contributions of the paper are as follows. The proposed system can process images from surveillance cameras in real-time by using down-sampling. It can effectively identify the identity of criminals by using a face tracking ID unit and minimizes prediction reversal by solving the congested embedding problem in the feature space that may occur when performing identification matching on a large amount of face embedding DBs. Additionally, the reliability of the identification results is complemented by an identification score accumulation method. In this paper, we prototyped the proposed system and experimented with the recognition model, achieving an accuracy of 0.900 and an F-1 score of 0.943. We also experimentally confirmed that the models proposed in other studies have higher performance when using the tracked instance-level face identification method proposed in this paper. It is expected that the proposed system can be used to locate criminals and protect national facilities, and such responses can quickly prevent accidents/incidents. The dataset and code are available at
https://github.com/aengoo/focusface