A Hierarchical Task Assignment for Manual Image Labeling

Manual image labeling (selecting an appropriate “category” for an image) is very tedious and time consuming especially when selecting labels from a large number of categories. In this study, we propose a hierarchical assignment of labeling tasks where the labelers recursively classify images in a category group into sub category groups, working on a single level at a time. This significantly makes each labeler's task easier, reducing the number of choices from 1,000 to 27 on average. In the user study, we compared our hierarchical assignment to a normal (non-hierarchical) assignment for a labeling task. The results show that the hierarchical assignment requires less total time to complete the labeling task. In addition, the learning effect in the labeling process is more profound in the hierarchical assignment.