Testing Aids and Their Impacts in Engineering Education: An Early Look at a Scoping Review

This research work-in-progress paper describes the initial steps towards a scoping review of knowledge-based testing aids in engineering education. The goal of this scoping review is to map the existing evidence and identify gaps in the limited and disjointed literature on this topic. In engineering education, the use of testing aids is commonplace, with even industry-standard examinations like the Fundamentals of Engineering exam utilizing a dedicated handbook. Despite the prevalent use of testing aids, a lack of consensus exists on best practices in this area. Two reviews exist in the literature, but whilst informative, they fall short by either focusing solely on the outcome of test performance or only comparing open-and closed-note conditions. Further, no literature from the last decade is considered in either review, and both reviews include a range of non-engineering disciplines with a heavy representation of medical education and psychology courses. Using an a priori protocol based on PRISMA-ScR guidelines, 7 academic databases were searched, including ERIC, Scopus, Web of Science, Engineering Village, and IEEE Xplore. The search yielded 236 journal articles and conference proceedings, of which 93 were identified as duplicates, 65 were excluded during title and abstract review, and 49 were excluded during full-text review, resulting in 29 relevant papers. This paper addresses 4 research questions: two are purely bibliometric, one explores the geographical distribution of studies, and the fourth examines the terminology used to describe knowledge-based testing aids. Analysis showed that there has been an increase in literature published in recent years (40^% were published since 2019), and that publications were evenly split between journal articles and conference proceedings (14 vs. 15). A majority of the papers (72%) explored exams with open-resource use, and most of those (21 out of 26) used the term “open book”. The findings from this paper will help refine a revised review protocol for the full scoping review. The full review will seek to answer two overarching research questions. These are 1) In what course contexts and examination conditions, and for which outcomes have testing aids been studied in the engineering education literature? and 2) What gaps in the engineering education literature exist?