Does Permitting Uncertain Estimates Help or Hinder the Wisdom of Crowds?

This paper adds to a growing body of research into the practical utility of using interval-valued (IV) response modes to efficiently capture richer quantitative data from people— e.g., through surveys. Specifically, IV responses offer a cohesive method of capturing uncertainty, vagueness, or range associated with individual quantitative responses. In turn, IV data provide a strong foundation for subsequent fuzzy set based modelling— e.g., using the Interval Agreement Approach. The present paper focuses on the impact of soliciting IV estimates upon accuracy of group perceptual judgements—the ‘Wisdom of the Crowd’. We report results from two empirical studies, examining the utility of IV data in the context of estimating specific (i.e., discrete point) ground truths, and directly comparing IV perceptual judgements (quantity estimates) against more traditional point estimates. There were two main hypotheses concerning the potential impacts of permitting uncertain (i.e., IV) estimates. First, it is possible that when specific predictions are required, permitting deliberately imprecise responses may reduce (prediction) accuracy versus forcing each respondent to provide their single ‘best guess’. Second, that capturing the uncertainty associated with individual predictions should permit improved aggregation of group estimates, through weighting individual estimates according to their certainty. We report findings from two studies designed to investigate these hypotheses, and outline proposals for future research in this area.