Reducing Marketplace Response Time by Scoring Workers

The growing smartphone user base has enabled new, paid, mobile crowdsourcing marketplaces, where individuals are paid to perform tasks using their mobile phones as they move around in their day-to-day lives. The massive crowdsourcing system serves task requesters who can proactively order data collection tasks from registered workers. The system should allocate incoming tasks to the better workers and still keep the costs of marketplace cloud backend low. We have built an online scoring mechanism that suits large-scale systems where each worker is evaluated continuously according to several parameters and an incoming task is allocated to top-grade workers. The quality of the scoring algorithm is determined by the average time a task remains in the system. The research evaluates the quality of the simple scoring algorithm that considers a worker's queue size at that instant (JSQ) and compares it with a simpler method that considers a worker's average queue length (AQL) with homogenous and heterogeneous workers. We have developed a simulator and identified real-world dataset parameters that can be used as inputs to the simulations. The simulation results show that allocating tasks by using simple shortest queue size achieves a better marketplace response time.