The Effect of Course Content Position on Student Attempts of Practice Problems in Introductory Biology

Since 2013, we have used all of the content from our Introduction to Biology Massive Open Online Course (MOOC) with students taking an equivalent on-campus, for-credit course in a mostly ungraded, blended manner. It appeared early on that students on campus watched the videos in high numbers, and more students answered practice problems from within video sequences than the (ungraded) problem sets or practice exams. As a follow-up, we asked, if the position of an ungraded problem impacts students' behavior to try practice problems. We hypothesized that placement of the ungraded problems in the course structure has a significant effect on the likelihood of a student attempting the problems. We predicted that ungraded problems placed immediately after a video segment within a sequence of videos would have a higher number of unique submissions compared to the same problem placed outside of the video sequence. To test our hypothesis, we performed A/B tests on the course content position of ungraded problems during both the fall 2015 and fall 2016 semesters. We positioned the ungraded problems in an appropriate video sequence or under the relevant problem set or exam available for practice in the course website. We found that more students attempted problems when the problems appeared in-between video segments for 34 out of 38 cases. Both sets of experiments (20 problems for 2015 and 18 problems for 2016) support our hypothesis that more students submitted answers to ungraded problems in the video sequence versus when we positioned the same problem in the course structure outside of the video sequence. These results indicate that the online course structure alone can influence student study behavior, a result potentially applicable to the design of blended learning experiences and MOOCs.