FedTA: Federated Worthy Task Assignment for Crowd Workers

Crowdsourcing is a promising computing paradigm for processing computer-hard tasks by harnessing human intelligence. How to protect online workers' privacy is a hindrance for deploying crowdsourcing in the real world. Attempts have been made to address this issue by injecting noise or encrypting sensitive data, which cause quality loss and/or heavy computation and communication load. In this paper, we propose an approach, called FedTA (Federated Worthy Task Assignment for Crowd Workers), to protect a crowd worker's private data while ensuring quality. FedTA trains a client model based on the private data and annotations owned by a worker and uploads client models to aggregate the server model, without leaking the privacy of task data. To account for the varying task distributions (i.e., non-i.i.d.) and error-prone annotations of tasks, it leverages the feature similarity and semantic similarity separately derived from client and server models on local tasks, to quantify the quality of annotations and clients. Based on those, it further introduces a task assignment strategy to notify the clients which tasks are worthy and suitable for annotations. This strategy can incrementally improve the performance of client and server models. At the same time, it disregards the unworthy tasks to save the budget and to avoid their negative impact. Experimental results show that FedTA can complete secure crowdsourcing projects with high quality and low budget.