Informing the Design of e-Learning Lessons on Data Science through Think-Aloud Experiments with Healthcare Professionals

Increasing data science adoption by healthcare professionals provides an opportunity to bridge the demand for analytics usage in clinical decision-making. One approach involves teaching data science concepts through an online education platform. We designed a think-aloud experiment in which we observed seven internal medicine residents complete an online interactive lesson using the Integrated Statistics Learning Environment (ISLE). The hour-long lesson focused on the predictive modeling lifecycle. Participants’ verbalizations and structured interactions with the platform were recorded. We observed a surprising misunderstanding of the lesson’s three main data science concepts. We also noticed a willingness to spend more time on familiar concepts and a comparative disengagement when seeing new topics. Many are amenable to the idea of learning data science through an online forum. However, it is important to first consult prospective clinician users prior to designing lessons to ensure that lessons align with clinical backgrounds, expectations and just-in-time learning needs.