Artificial Intelligence Literacy Test for Early Adolescents: Design Principles, Content Validation, and Pilot Testing

With the rapid advancement of artificial intelligence (AI) technologies and their increasing integration into daily life, the importance of AI education at the K-12 level has gained significant attention. Consequently, there is a growing need to develop valid assessment tools to measure students' understanding of AI literacy. While several efforts have been undertaken to design AI literacy assessments, there remains a lack of well-validated tests specifically tailored for students aged 10 to 15. To address this gap, this study developed the Artificial Intelligence Literacy Test for Early Adolescents (AILt-EA). Evidence-Centered Design (ECD) was employed for the test design. A rigorous validation process was undertaken, including expert reviews and cognitive interviews with students, to ensure content validity, and revisions were made based on the feedback. Then, a pilot test was conducted with 151 seventh-grade students at a public middle school in northern China. Item analysis was performed, and decisions regarding test improvements were made to enhance the validity of the test. This study provides a valid and reliable tool for assessing AI literacy, offering a foundation for future research and teaching practices in K-12 AI education.