Comparing three methods of measuring human judgment performance in an air traffic conflict prediction task

An experiment using a simulated air traffic conflict prediction task with sensor noise was conducted with one between subjects manipulation and the sensor noise level. Measures from signal detection theory, fuzzy signal detection theory, and judgment analysis were calculated and analyzed. Gender, noise, and session (experience) effects and their two-way interactions were investigated. As different effects were significant for different measures, analysts should take care in relying on the results from any single measure when analyzing human judgment performance.