Engineering Ph.D. Students’ Research Experiences: A Think-Aloud Study

This Research Work in Progress Paper describes the initial development and validation for using a new assessment instrument to understand how the research experiences of engineering Ph.D. students shape them as future professionals. Little is understood about how engineering Ph.D. students are prepared for professional practice through their research experiences, yet this is of growing concern to the engineering education community. This paper describes how assessment items were developed and the collection of initial validity evidence through a structured think-aloud protocol where engineering Ph.D. students read and interpreted the items, verbalizing their thinking as they completed the assessment. Two rounds of think-alouds were completed with engineering Ph.D. students. The first round was conducted with seven students and resulted in modification of twenty-eight of the thirty assessment questions, including minor word changes to twenty-two questions and relatively major rewrites to six questions. The second round was conducted with five students and resulted in modification of twelve of the thirty assessment questions, with only minor word changes to the questions. The evidence collected shows that the vast majority of the assessment questions were understood as intended, and the assessment is ready for pilot data collection through a survey.