Confirmation Bias in Complex Analyses

Most research works investigating the confirmation bias has used abstract experimental tasks where participants drew inferences from just a few items of evidence. The experiment reported in this paper investigated the confirmation bias in a complex analysis task that is more characteristic of law enforcement investigations, financial analysis, and intelligence analysis. Participants were professionals, half of whom had intelligence analysis experience. The effectiveness of a procedure designed to mitigate the confirmation bias, called analysis of competing hypotheses (ACH), was tested. Results showed a confirmation bias for both experience groups, but ACH significantly reduced bias only for participants without intelligence analysis experience. Confirmation bias manifested as a weighting bias, not as an interpretation bias. Participants tended to agree on the interpretation of evidence (i.e., whose hypothesis was supported by the evidence) but tended to disagree on the importance of the evidence—giving more weight to the evidence that supported their preferred hypothesis and less weight to evidence that disconfirmed it.