Selective face de-identification scheme using multiple face recognition and classification techniques

Social media is rapidly growing through platforms that provide live-streaming service, video sharing service, etc. But, as the social media grows, the cases where faces of people in the media are indiscreetly exposed are also increasing. Personal sensitive information may be included in a photo and video, and representative data that can identify an individual is face. So, various face de-identification methods have been proposed. However, most face de-identification methods have tried to obfuscate every face in an image or video and tend to focus on minimizing the loss of facial features due to obfuscation for utilizing the rest of facial features as data except for features that can identify individuals. Therefore, those methods are not adequate for face de-identification in social media service because main characters' faces like media producers, creators or actors need to be exposed on purpose, whereas the other targets' faces like pedestrians or crowd should not be exposed in the videos. Selectively obfuscating targets' faces by classifying all faces in the video into exposing targets and obfuscating targets is necessary for improving the privacy in social media service. In this paper, we propose a selective face de-identification system that exposes main characters' faces and obfuscates the other targets' faces by recognizing and classifying all faces appeared in a video. Our system detects and tracks multiple faces using a deep learning-based face recognition, and classifies main characters' faces and the obfuscating targets' faces, and finally de-identifies only the faces of the obfuscating targets on purpose. We also propose an efficient pipelining scheme for batch processing of face de-identification that rearranges the order of faces to be identified in each frame by matching faces existing in continuous frames of a video.