An Exploratory Study on the Impact of AI tools on the Student Experience in Programming Courses: an Intersectional Analysis Approach

This work-in-progress paper presents a study that sheds light on the concerns that students may not develop sufficient programming skills and as a result, be less competent with the use of ChatGPT. The potential benefits for students are significant: Access to ChatGPT increases the ability for students to work constructively on their own schedule. The ease of use of ChatGPT may engage students who might otherwise hesitate in asking for support. Before these tools can be meaningfully introduced into a course, work must be done to study the impact of these AI tools on a student's ability to learn. In this study, participants are recruited from introductory Java programming courses at a large public university in the United States. This paper presents preliminary findings from a mixed method study design that consists of a pre-task assessment quiz; and a programming task in one of three conditions: (1) with no external help, (2) with the help of an AI chatbot, or (3) with the help of a generative AI tool like GitHub Copilot; followed by a post-task assessment and an interview on their experience and perceptions of the tools. Our preliminary findings describe our data collection, thematic analysis of the students' prompts and chatGPT responses, and a summary of the experience for 3 students. Our findings demonstrate a range of students' attitudes and behaviors towards chatGPT that provides insight for future research and plans for incorporating such AI tools in a course.