WIP: Engineering Class Students' Epistemic Cognition when Interacting with Generative AI

This work in progress belongs to the innovative practice category. Nowadays, generative AI (also known as GenAI) can produce novel data samples that closely resemble authentic datasets. The advent of large language models (LLMs), in particular, has caused a huge interest in utilizing GenAI within and beyond the realm of higher education. However, little about engineering students' views and behaviours related to knowing and knowledge when using GenAI, such as ChatGPT, is known. In this WIP, we have engaged a class of N = 37 engineering students taking a postgraduate course titled “Social Media Analytics”. They were required to write essays related to their course learning in the form of blog posts. They were required to use LLM tools, such as ChatGPT, to assist their writing processes. Their GenAI usage was guided by the cognitive-agent approach, Search Tree, Analyze and Repair, and Selection (STARS), while STARS was proposed by Kirk et al. in AAAI 2024 to extend and complement prompt engineering. In addition, the participants were invited to fill in the Epistemic Cognition Inventory (ECI) questionnaire to associate five aspects of epistemic cognition (EC) with their writing experience. It is confirmed in our results that students' EC, i.e., their beliefs related to knowledge and knowing, significantly predict their prompting engagement and academic performance. However, students' academic performance is found to be significantly and negatively associated with their preference for GenAI usage. Here, we have uncovered engineering students' EC when interacting with generative AI, an area where little has been known so far. Our findings also suggest that proper use of GenAI prompting might promote engineering students' EC and, therefore, engineering learning.