Crowdsourcing System Management for Activity Data with Mobile Sensors

In this paper, we propose crowdsourcing system management for activity data with mobile sensor devices such as smartphones and accelerometers in human activity recognition. We describe the design rationale of the systems and proposed crowdsourcing approach, analyze the collected data through statistics, and application of an activity recognition method. We developed the systems to efficiently collect activity data with mobile sensors from workers in crowdsourcing services. By using the systems, we gathered 1,749 activity data from 32 participants through Amazon Mechanical Turk. We show a reference accuracy of recognition of activity using the obtained data, the performance of activity recognition attained the balanced accuracy score of 0.55, the F-measure of 0.96, the precision of 0.83, and the recall of 0.76 on average by exploiting a random forest classifier for classification study. We found still a challenging field for activity data collection in crowdsourcing services, including activity classes, timing, and activity recognition performance.