Using Text Analytics on Reflective Journaling to Identify Cultural Capitals for STEM Students

Alma project focuses on retention of Historically Underrepresented (HU) students in STEM fields by affirming their cultural capitals (skills, knowledge, communities, and abilities that students possess). Students are asked to write reflective journals on prompts such as `Why am I here?' to reminds them of their cultural capitals, and career aspirations. These essays are then analyzed by the Alma team to identify mentions of cultural capital themes (CCT) such as Attainment (describes a tangible goal), or First Generation (identifies student being first in family to attend college). However, detecting these CCTs in student essays is a labor intensive task. Our work offers a computational solution that is scalable and effective for this problem. We propose TACCTI (pronounced as Tacit), Text analysis and mAchine learning for Cultural Capital Theme Identification - a computational framework that employs latest developments in natural language processing and machine learning to identify instances of cultural capital themes in student essays. An empirical evaluation using Logistic Regression, Random Forest, LSTM, and BERT algorithms for the task of detecting presence (or absence) of Attainment CCT provides F1 scores of 0.82, 0.81, 0.84, and 0.92, respectively. Classification models for detecting First Generation CCT demonstrate that when limited training data is available, the traditional classification algorithms outperform deep learning based approaches - Logistic Regression and BERT models offer effectiveness of 0.91 and 0.83, respectively. Additional analysis demonstrates that word embeddings and named entity type of features (signals) have the highest influence on the performance of the classification models.