Performance of M-ary soft fusion systems using simulated human responses

A major hurdle in the development of soft and hard/soft data fusion systems is the inability to determine the practical performance gains between fusion operators without the burdens associated with human testing. Drift diffusion models of human responses (i.e., decision, confidence assessments, and response times) from cognitive psychology can be used to gain a sense of the performance of a fusion system during the design phase without the need for human testing. The majority of these models were developed for binary decision tasks, and furthermore, the few models which can operate on M-ary decision tasks are yet unable to generate subject confidence assessments. The current study proposes a method for realizing human responses over an M-ary decision task using pairwise successive comparisons of related binary decision tasks. We provide an example based on the two-stage dynamic signal detection models developed by Pleskac and Busemeyer (2010) where subjects were presented with a pair of lines on a computer screen, asked to determine which of two lines was the longest, and to assess their confidence in their decision using a subjective probability scale. M-ary human opinions were simulated for this line length task and used to assess the performance of several fusion rules, namely: Bayes' rule of probability combination, Dempster's Rule of Combination (DRC), Yager's rule, Dubois and Prade's rule (DPR), and the Proportional Conflict Redistribution rule #5. When taking source reliability into account in the combination, Bayes' rule of probability combination and DRC exhibited the most accurate performance (i.e., the largest amount of specific evidence committed towards the true outcome) for this task. Yager's rule and DPR exhibited inferior performance across all simulated cases.