Webly-supervised visual concept learning with cardinality guided instance mining and clustered multitask refinement

Conventional image classification and object detection methods depend on manual annotations, such as image-level labels and bounding boxes. However, the acquisition of such annotations for millions of images is trivial. This paper addresses the problem of webly-supervised visual concept learning, and develops an automatic algorithm using parallel text and visual corpora to discover informative visual patterns from the web images. Based on the mined patterns, a cardinality-guided multiple instance learning algorithm is designed to establish the link between the image patterns and the literal concepts. Furthermore, due to the diversity of visual concepts, we perform clustered multitask refinement on the learned concept classifiers to enhance their generalization capability via a clustered regularization. Experiments demonstrate the superiority of the proposed method over traditional approaches.