Towards EEG-based Performance Assessment in Dataset Annotation Tasks

Artificial intelligence is more and more adopted to complement human activity in solving complex tasks in several domains, including healthcare, security, finance, and automation. In order to be effective, several artificial intelligence tools rely on large training sets of carefully annotated data. Since labeling is mostly performed manually, it is a costly and error-prone process. Hence, there is increasing interest in devising innovative tools to support the annotation task. In this paper, we report an initial investigation on the application of EEG data mining for evaluating the performance of humans carrying out image annotation tasks. Our approach relies on a cheap portable EEG sensor and on supervised learning methods. We collected a dataset from five volunteers, and performed an initial evaluation of our technique. The achieved results are promising, and pave the way to several research directions. To the best of our knowledge, our work is the first one applying EEG data mining for assessing the performance of labelers.