Design an Assessment for an Introductory Computer Science Course: A Systematic Literature Review

This Research Full Paper presents a literature review on introductory CS assessments. As Computer Science (CS) becomes increasingly popular in the world of big data, more college students are taking introductory CS courses to follow the trend. However, students have different levels of exposure to CS when they start CS in college, varying from no experience to years of coding experience. This disparity in the amount of exposure can pose many problems. In the same intro-level class, some students may think they do not learn anything new, while others believe the professor goes through concepts so fast that they cannot follow the pace. CS professors may also feel confused about how to control the pace and content of introductory courses. Therefore, CS departments need an assessment to measure students’ CS knowledge and then assign them to the appropriate CS introductory courses. Though a vast number of research papers discuss CS introductory courses, few of them focus on developing and validating assessments that evaluate students’ background knowledge (i.e., concept inventory). This paper aims to conduct a systematic literature review to gain an overview of the most important introductory CS contents that need to be assessed, the methods to evaluate students’ understanding of introductory CS concepts, and some existing assessments/concept inventories. By reviewing relevant papers over the last 15 years, we selected 48 papers that are related to assessments and corresponding introductory courses. We collected assessment information and made tables of contents, question types, programming languages, and origins to analyze data. We found that iterations, conditionals, and variables are the most popular assessment contents. Multiple-Choice and Code Writing questions, which are more popular than Code Explaining questions, are the preferred ways of assessing students’ knowledge. Scratch is commonly used as a CS0 assessment language, while Python, Java, C, and Pseudo-codes are employed in CS1 assessments. FCS1, SCS1, and BDSI are existing assessments/concept inventories for CS1 or CS2, but there is not yet a validated assessment or concept inventory that encompasses introductory concepts of CS0 (computational thinking), CS1 (fundamentals), and CS2 (data structure). Designing a comprehensive CS0/CS1/CS2 assessment by using the information collected from our systematic literature review would help students choose the right course and benefit educators and researchers for general use.