A Comparison Between Transformers and Foundation Models in Sentiment Analysis of Student Evaluation of Teaching

Student evaluation of teaching (SET) serves as a crucial tool for improving educational quality, enabling students to articulate their opinions about instructors. However, manually evaluating student feedback is time-consuming, subjective, and prone to error. Sentiment analysis, which automatically classifies texts using computational algorithms, presents a promising alternative for this task. In this work, we conduct a comparative analysis of sentiment analysis on SET between three Transformer networks and three Foundation models on a dataset from an Ecuadorian university. Our experiments demonstrate that Transformer models trained on the dataset of interest have a better overall performance than general-purpose Foundation models. Furthermore, among the models examined, DistilBERT emerges as the top performer, achieving an accuracy of 84.90% and an F-1 score of 0.836. In comparison, among the Foundation models, Google Bard achieves the highest accuracy and F-1 score with 78.3% and 0.767, respectively. This work contributes valuable insights to the realm of higher education evaluation, showcasing the potential of advanced NLP techniques to expedite and enhance the SET process, ultimately paving the way for continuous improvement in educational settings.