Methods matter: Contrasting undergraduate research experience outcomes based on surveys and interview methods

Undergraduate students who engage in research can realize a variety of personal and professional benefits. Studies of undergraduate research experiences in engineering have largely relied on survey methods, which can be subject to positive response bias due to acquiescence and/or social desirability. This research compared two convenience sources of information to determine the impacts of undergraduate research experiences for engineering students. Survey findings from 8-years assessing a summer REU site focused on environmental engineering (n=85) tended to be overwhelmingly positive based on Likert-type responses to pre and post survey questions. Open-ended responses were enlightening in regards to the most valuable aspect of the summer REU experience for the students. The second data source were yearly student interviews not explicitly targeted to study research experiences; nine engineering students discussed their research experiences. The interviews yielded more balanced discussions on the negative and positive aspects of research experiences. The immersive summer program that included cohort elements and intentional scaffolding generally provided different experiences compared to more individualized research during the academic year. The interview findings point to the benefits of rich research methods and raise concerns about placing too much weight on assessments based on responses to agree-disagree type survey items.