Object Detection with Task Description Only

Deep Convolutional Neural Network models achieve state-of-the-art performance on object detection, but it is driven by large labeled data sets. For general practical object detection problems, the existing public data set is no longer applicable, and there is no ready-made data set to solve the problem, usually only the description of object detection task is available. The method of collecting data sets from real world is really time consuming and unreachable in some special scenarios. In this paper, we define this problem as a new object detection problem named Task Description Object Detection, and present a synthetic image-based method to solve this problem. We design a system to generate annotated synthetic images quickly and inexpensively according to the task description. What's more, we propose a novel layer called SOMConv for object detection network to adapt the object detection model trained with the synthetic data sets to detection in real world. Our experiments evidence the effectiveness of our approach on real-world object detection problems, which only has the description of the object detection problem and no real images.