Efficient Worker Assignment in Crowdsourced Data Labeling Using Graph Signal Processing

The first step in solving a classification problem is to collect and label a sufficient amount of training data. Given the time and cost associated to data labeling, crowdsourcing systems (e.g., Amazon Mechanical Turk) are often used. However, one of the key disadvantages of crowdsourcing systems is the presence of spammers or workers who are not as skilled or careful, thus leading to many false labels being assigned. This paper addresses this problem by proposing a novel algorithm based on graph signal sampling theory, which optimally assigns data to different workers for labeling by taking into account the expected quality of labeling provided by each worker. Our simulation of the labeling process using these schemes shows that the classification error can be reduced significantly with respect to a random assignment of workers.