By Topic

Iterative processing algorithm to detect biases in assessments

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

1 Author(s)
R. C. Woods ; Dept. of Electr. & Comput. Eng., Iowa State Univ., Ames, IA, USA

In order to assess a large number of diverse projects as fairly and rapidly as possible, a procedure often adopted is to use a panel consisting of a large number of experts, only a small number of whom assess each project. Since no one expert assesses all the projects, conscious or unconscious bias regarding overall standards by any expert will advantage or disadvantage the projects assessed by that particular expert. This paper presents an iterative algorithm that has been used successfully to detect and correct for such biases. Each expert's assessments are modeled as differing from the ideal as a result of a shift of mean and having a standard deviation that is too low or too high. This model is used in conjunction with the concept of "paired assessments" to account for individual projects being of unusually high or low quality and so to evaluate the discrepancy from the ideal marks. The same computer program also has applications in the peer-review or expert-evaluation of research proposals, and any other situation involving subjective assessments by a restricted number of persons.

Published in:

IEEE Transactions on Education  (Volume:46 ,  Issue: 1 )