AH-CoLT: an AI-Human Co-Labeling Toolbox to Augment Efficient Groundtruth Generation

Two critical aspects in building successful AI-based visual recognizers are having access to enough representative labeled data and creating a robust learning paradigm. The need for large scale training set is clearly understood in supervised learning models especially with deep structure; however, even for semi-supervised or unsupervised algorithms having a large set of labeled data is crucial for quantitative and comparative model performance evaluation. Reaching to the limits of manual annotation by human experts or via crowd sourcing, for the next big breakthroughs in the machine learning domain we need much more data/label pairs. To address this need, our AI-Human Co-Labeling Toolbox (AH-CoLT) presents an efficient semi-automatic groundtruth generation framework for unlabeled images/videos. AH-CoLT enables accurate groundtruth labeling by incorporating the outcomes of state-of-the-art AI recognizers into a time-efficient human-based review and revise process. In this paper, we evaluated the performance of the proposed AH-CoLT framework on generating groundtruth labels on human pose images. The resultant pose labels using our toolbox are shown to be more accurate and achieved in a more time-efficient way, when compared to the groundtruth pose labels generated by crowd sourcing.