Scene-Specific Anomalous Relationship Detection Using Scene Graph Summarization

This paper defines a novel task, Scene-Specific Anomalous Relationship Detection (SARD), designed to detect anomalous relationships between objects within a given scene by analyzing an image collection of that scene. The definition of “anomalous” can vary from scene to scene. For example, placing a cup near a book (“cup-near-book” relationship) is typical in general contexts. However, it may be considered anomalous in a specific scene, such as a library where drinking is usually prohibited. By observing a specific scene for a while, humans can discern the underlying rules or commonsense contexts within that scene. Building on this insight, this study proposes a method for SARD based on scene graph summarization that can capture scene-specific contexts. The method generates a scene graph from each image and then combines them into a summarized scene graph, retaining the occurrence count for each relationship. It calculates the anomaly score of each relationship within the summarized scene graph based on its rarity and outputs ranked relationships. To confirm the performance, this paper introduces a new dataset for the SARD task. By comparing the proposed method with baseline methods on the dataset, the proposed method achieves an increase of 19.77 in the AUC of Recall@k. We provide the code and dataset on https://github.com/marow17623/SARD.git.