Visual Detection of Unknown Objects in Video Games Using Qualitative Stability Analysis

Many current computer vision approaches for object detection can only detect objects that have been learned in advance. In this paper, we present a method that uses qualitative stability analysis to infer the existence of unknown objects in certain areas of the images based on gravity and stability of already detected objects. Our method recursively searches these areas for unknown objects until all detected objects form a stable structure or no new objects can be identified anymore. We evaluate our method using the popular video game Angry Birds. We only start with detecting the green pigs and are able to automatically identify and detect all essential game objects in all 400+ available levels. All objects can be accurately and reliably detected. Our method can be applied to other video games where objects obey gravity and are bound by polygons.