Observer Reliability and Human Inference

This paper presents a formal analysis of the problem of determining the inferential impact of the information in a composite report from a collection of unreliable observers or sensors. Each sensor reports one of a finite number of possible states of a data system linked probabilistically with an ``objective system'' whose condition is to be inferred from the data state. The principal assumptions are that the sensors do not ``collaborate'' in making their reports and that their reports are conditioned only by the existing data state and not by the actual, unobservable state of the objective system. Use of the notion of conditional independence to express these assumptions gives the analytic expressions a tractable form which sheds light on various inference issues. The paper also briefly discusses current empirical research on the question of how well people actually adjust the impact of inferential evidence to correspond to the unreliability of the sources of information.