Aggregating Unstructured Submissions for Reliable Answers in Crowdsourcing Systems

In crowdsourcing systems, requesters assign tasks to multiple workers for obtaining reliable solutions. Task aggregation is performed on submissions received from crowd workers. Existing aggregation methods focus on structured submissions without accounting reliability factor of workers. Hence they are not suitable for aggregating unstructured answers. This paper proposes a novel task aggregation approach for a generic crowdsourcing tasks using an iterative probabilistic model. We make use of the reliability parameter and expertness of workers along with similarity information and requesters feedback. Experiments on the empirical data demonstrate that our scheme yields better results compared to existing state-of-the-art approaches.