Using frame-of-reference training to improve the dispersion of peer ratings in teams

An engineer's ability to work in teams is critical to their engineering career and often a significant factor in a corporate hiring process. Recognizing this need, and motivated to demonstrate outcomes required for accreditation by ABET, most U.S. undergraduate engineering programs include team-based courses in their curricula. Hundreds of these engineering programs use the CATME application (the Comprehensive Assessment of Team-Member Effectiveness), which asks students to assess the teamwork behaviors of their peers. Our goal was to determine how effectively engineering students rated their peers and whether a proven method for improving the use of a rating schema, Frame of Reference training, would improve engineering students' peer rating behavior. We discovered that engineering students do not materially change their peer ratings based on their teaming experiences and we verified that repeated Frame of Reference trainings have a significant impact on peer rating dispersion and potentially the quality of engineering students' peer ratings.