Analysis of Vlog Transcripts using Topic Modeling, Summarizer, and Cluster Analysis

This paper demonstrates the use of natural language processing (NLP) models to analyze qualitative data for engineering education research purposes. Three NLP models are applied: topic modelling, which identifies salient keywords in text; summarizer, which extracts and concatenates sentences with unique meaning from a text; and cluster analysis, which groups texts together based on similar word sequences. The study applied these techniques to short video logs, or vlogs, collected as part of a case study of undergraduate engineering students' exposure to a social innovation curriculum. The curriculum aimed to teach students to use their engineering skills to identify and address issues that cause suffering for marginalized communities. In the vlogs, students responded to prompts asking them to describe what social issues they are passionate about, how they relate to these issues, and how they would propose exploring and addressing those issues. Vlogs were transcribed, pre-processed, and then examined with each technique to identify patterns within and across the prompts. Widely-recognized limitations of NLP techniques included the potential loss of important contextual information and the dependence on large volumes of data to produce valid and reliable results. Despite these limitations, the combination of three techniques was effective for locating high priority transcripts within the data corpus, identifying themes within and across vlogs, and supporting longitudinal analysis of student responses. Previous literature has documented the utility of topic modeling and other NLP techniques to analyze large volumes of written text on, for example, course evaluations or student writing assignments. Importantly, this study demonstrates the novel and meaningful application of topic modelling, summarizer, and cluster analysis to analyze a relatively small corpus of transcript data. Given these results, we are optimistic about the potential for NLP approaches to complement other analysis techniques and make the analysis of transcript data more efficient and feasible.