How do you Perceive Differently from an AI — A Database for Semantic Distortion Measurement

Artificial intelligence (AI) is enabling the automated analysis of large amounts of image/video data, boosting the speed of multimedia data processing remarkably. Meanwhile, Image Quality Assessment (IQA) plays an important role in developing automatic analysis methods. To ensure the effectiveness of AI, images in multimedia applications should be considered for visual examination by both human and machine. Therefore, it is significant to understand the differences between human's and AI's perception of semantic distortion. However, little work has been done due to the lack of data from human on the semantic level. In this paper, we first propose a semantic database (SID) based on the surveillance scenarios, by collecting subjective average recognition rates of 3 semantic targets (face, pedestrian, license plate) with 3 types of distortion (JPEG Compression, BPG Compression, Motion Blur). Then, we present a detailed analysis of how human and AI perceive semantic distortion differently. Experimental results show that AI is stronger in tolerance to distortion than human beings on average, while weaker at generalization and stability. It is also implied in the experiments that existing IQA methods are not effective enough at judging the semantic distortion.