Improving Accuracy of Self-and-Peer Assessment in Engineering Technology Capstone

Contributions: This article presents the findings of a study on the impact of instructional interventions (training and additional requirements and prompts) to improve the quality of students’ self-and-peer assessment in a capstone engineering technology course. Background: Historically, students in this course inflated rating values in comparison to the rubric, which specified what behaviors would “meet” or “exceed” expectations. Intended Outcomes: This study addresses the following research question: what is the impact of the interventions on student self-and-peer assessments? This is explored through comparing the distribution of peer-assessment and self-assessment ratings between different iterations of the course design, as well as the distribution of ratings across the duration of the year-long project experience. Application Design: Several instructional interventions aimed at self-and-peer assessment training were developed and implemented to improve the accuracy of student ratings over the course of four years. Interventions included adding lectures about the comprehensive assessment of team member effectiveness (CATME) scale; encouraging students to review the CATME rating scale prior to self-and-peer assessment; implementing a new component that is a team communication plan into the team reports written at each of six “gates” across the year-long project; requiring students to define the levels in the rating scale for their own team; and providing ongoing faculty feedback on team communication plans, team defined rating scales, and overall team performance. Findings: A comparison of student ratings across and within years indicated that the interventions were successful in moving the distribution of ratings toward the center, or “meets expectations.” Findings align with the existing literature and may be relevant across disciplines.