Improving Image Search with PHETCH

Keyword-based image search engines are hindered by the lack of proper labeling for images in their indices. In many cases the labels do not agree with the contents of the image itself, since images are generally indexed by their filename and the surrounding text in the Web page. Another popular approach to image search, content based image retrieval, suffers from a gap between the available low level data and the semantic needs of user searches. To overcome these problems we suggest human annotation of images with natural language descriptions. To this end we present Phetch, an engaging multiplayer game that allows people to attach accurate explanatory text captions to arbitrary images on the Web. People play the game because it is fun, and as a side effect we collect valuable information that can be applied towards improving image retrieval. Furthermore, the game can also be used for other novel applications.