Understanding Fatigue and its Impact in Crowdsourcing

Crowdsourcing has been widely considered as a rapid and cost-effective way to complete tasks by harnessing human intelligence over the Internet. In recent years, increasing attention has been drawn to the issues of crowdsourcing fatigue and worker engagement, in a hope to enhance output quality by improving worker experience. However, while crowdsourcing fatigue issue is commonly recognized, there is still lack of systematic study to understand how exactly fatigue occurs and how it affects crowdsourcing. This paper presents an experimental study to investigate these issues by employing quantitative and qualitative means with an image-tagging project on a crowdsourcing platform. Our study confirms that feeling of fatigue is common and significant after a batch of tasks, however, work efficiency is still increased even after workers feel fatigue, largely due to factors such as increased familiarity, skill, etc. Based on our quantitative and qualitative results, we also suggest implications for design to address issues of fatigue, and improve crowdsourcing work experience and output.