Anomalies detection in evaluation of essay type answers by multiple evaluators

In this paper, we present a methodology to detect suspected anomalies in evaluation of essay type answer books based on relationship between marks awarded and symbols used during evaluation. We identified dominant symbols used by evaluators and their relationships with marks awarded based on analysis of evaluated essay type answer copies. Further, for each evaluator relationship between symbols used and marks awarded is distinct. We identified outliers for each evaluators and each subject based on relationship between marks awarded and tick symbol and termed them as suspected anomaly. All grievances of students related to evaluation were in the set of identified outliers.