A Partial Contour Similarity-Based Approach to Visual Affordances in Habile Agents

In a typical tool use task, we can view both the relationship between the agent and the tool and the relationship between the tool and the target in terms of affordances. One set of affordances relates to the ability of the agent to manipulate the tool, while a second set of affordances relates to the ability of the agent to manipulate the target by means of the tool. In both cases, effective tool use is facilitated by the coupling of one object to another: agent-to-tool-to-target. In this paper, we focus on the visual identification of such affordances via contour similarity. Objects with complementary contour segments can fit together, which suggests possible opportunities for effective interactions. We present a system for the identification and evaluation of partial contour-based matches and analyze the system's behavior. We propose a set of sample tool-use scenarios as part of our analysis. We demonstrate the use of the system in providing guidance to an autonomous robotic agent performing tool selection tasks.