Providing Semantic Annotation for the CMU Grand Challenge Dataset

Providing ground truth is essential for activity recognition for three reasons: to apply methods of supervised learning, to provide context information for knowledge-based methods, and to quantify the recognition performance. Semantic annotation extends simple symbolic labelling by assigning semantic meaning to the label, enabling further reasoning. In this paper we present a novel approach to semantic annotation by means of plan operators. We provide a step by step description of the workflow to manually creating the ground truth annotation. To validate our approach we create semantic annotation of the CMU grand challenge dataset, which is often cited but, due to missing and incomplete annotation, almost never used. We evaluate the quality of the annotation by calculating the interrater reliability between two annotators who labelled the dataset. The results show almost perfect overlapping (Cohen's κ of 0.8 between the annotators. The produced annotation is publicly available, to enable further usage of the CMU grand challenge dataset.