Analyzing the Competitive Mathematical Problem-Solving Skills of ChatGPT

The recent emergence of ChatGPT (CGPT) has been considered a machine-learning breakthrough by many for its ability to “speak” like a human and its ability to explain, tutor, and assist. In this study, the ability of CGPT to solve competitive mathematics problems from the AMC (American Mathematics Competitions) with varying type and complexity was evaluated. The problems were categorized into five fields of competitive mathematics specified by number theory, geometry, algebra, combinatorics, and computation, and then further categorized into three different complexity levels (easy, normal, and hard). Results indicated that CGPT performed the best in the number theory and computation categories but did not perform well in subjects such as geometry and combinatorics. On average, 50 to 60 percent of the problems were correctly solved by CGPT; however, it could not solve most problems of higher complexity. This point indicated that CGPT failed to correctly apply concepts crucial to solving specific problems. Therefore, CGPT was found to be weak in skills involving spatial reasoning and problem-solving.