Practical Active Learning Stations to Transform Existing Learning Environments Into Flexible, Active Learning Classrooms

Contribution: Practical active learning stations (PALSs)-equipped classrooms function similar to prototypical active learning classrooms (ALCs). They support student collaboration and active learning pedagogies but at a fraction of the cost. Background: Active learning pedagogies and active learning technology are revitalizing STEM education and their use has led to an increase in student performance and satisfaction with the learning environment in postsecondary settings. An obstacle to increasing access to ALCs is the cost of constructing such learning environments. To address this challenge, a means to retrofit an existing computer laboratory into an ALC by making use of economy hardware and open-source software was devised. Intended Outcomes: In the context of an introductory sequence of programming courses (i.e., CS1 and CS2), students in a PALS-equipped classroom would perform as well as students in a prototypical ALC. Application Design: A quasi-experimental study was employed to compare the overall student performance across learning environments. Student performance was measured by the final exam score and overall course score. Throughout the study, the PALS-equipped classroom was paired five different times in head-to-head comparisons with either a prototypical ALC or a traditional classroom. Findings: The focus of the study was the potential effects of classroom type on students’ final exam score and the overall course score. A statistically significant effect was found for only one measure, which was that students in the PALS classroom in CS1 scored higher on their overall course score even when accounting for demographic differences and the pretest measure. There were no other significant effects for classroom type, either on the final exam score for either course or the overall course score in CS2.


I. INTRODUCTION
A CTIVE learning and active learning classrooms (ALCs) have reinvigorated STEM education at the post-secondary level, increasing student performance and Manuscript received January 12, 2020; revised May 27, 2020 and June 29, 2020; accepted July 11, 2020. Date of publication August 5, 2020 satisfaction over more traditional delivery modes or learning environments [1]- [3]. Several designs for ALCs have been developed and studied; arguably, the most well-known designs include SCALE-UP [4], TEAL [5], and REAL [6]. Common to these designs are circular or U-shaped tables placed around the periphery of the room. Displays are present at each table and students can connect their devices to share content locally or with the entire class by making use of large displays around the classroom. To further facilitate communication, ALCs are often equipped with whiteboards, microphones, and video capture devices. Given the reported benefits of these early designs, they became a model for classrooms designed to support student interaction and collaboration. An enduring challenge with such designs has been the cost, with reports of constructing or retrofitting existing traditional classrooms to prototypical ALCs easily reaching hundreds of thousands of U.S. dollars [3], [7], [8]. These costs may place these proven types of learning environments beyond the reach of many institutions of higher learning.
To increase access to active learning spaces, the broader computer science education community has responded with several initiatives. In recent years, there have been a number of studies on flexible learning spaces [9]- [14]. These spaces attempt to mimic some aspects of an ALC and are often achievable at a reduced cost. Many flexible learning spaces opt not to replicate the screen-sharing functionality of prototypical active learning environments. Removing such functionality reduces the cost and complexity of the learning environment and may make sense in some disciplines. Yet, given the types of digital artifacts (e.g., programs) common of students in computing and engineering fields, the ability to share screens aids in collaboration and supports interactions common to the discipline (e.g., pair programming). As a result, it would be ideal if economical ALCs supported more of the affordances of prototypical ALCs.
In response to the need, an economical system to convert a traditional computer laboratory into a full-fledged ALC was developed. The system is built from commodity hardware and open-source software and leverages resources commonly available to computer science and engineering programs. At the heart of the system are student stations called PALS (short for practical active learning stations). To study the effectiveness of the system, a computer laboratory was converted into a PALS-equipped ALC. Over the course of a three-year study, the PALS-equipped classroom was compared to a prototypical, state-of-the-art ALC in a CS1 and CS2 setting. Reported here is the effect of learning environments on the overall performance, as measured by the final exam score and the overall course score. This study was guided by the following research question: how does the classroom environment affect students' performance in CS1 and CS2 as measured by the final exam score and the overall course score?

A. Description of Learning Environments
To evaluate the effectiveness of an economical ALC, an existing computer lab was transformed to function similar to a prototypical, state-of-the-art ALC. This transformation was accomplished by rearranging the existing furniture from row seating to group seating with tables stationed around the periphery of the classroom. Three large LCD monitors were mounted on wheeled carts and placed around the classroom. To support local collaboration, practical active learning stations (PALS) were positioned two to a table. PALS consisted of an LCD monitor equipped with an HDMI capture device and back-mounted micro-ITX PC. The PC ran a Linux stack capable of consuming HDMI input and mirroring it locally. Periodic screenshots of local content were sent to an instructor station and upon the direction of the instructor, local content could be mirrored on the larger LCDs for class-wide consumption. Several small whiteboards were affixed to the walls and available for student use. Additional details with respect to the construction of the PALS system and the transformation of the classroom are available online and in the previous work [15]- [17]. For this study, the relevant affordances of the PALS-equipped classroom include the sharing of digital artifacts (locally and class wide) and group-centric layout of the classroom and whiteboards. Fig. 1 shows the economical, PALS-equipped ALC.
The additional learning environments utilized for this study were a prototypical, state-of-the-art ALC, and a traditional classroom. The ALC had nine large U-shaped tables positioned around the periphery of the classroom. Each table sat up to seven students and shared one, medium-sized LCD at the end of the table. Large LCDs were placed high upon the walls around the classroom. Handheld whiteboards were available and stored on the wall near each table. An instructor station was located at the center of the classroom. Although not utilized in this study, the ALC was also equipped with microphones at each table and video cameras that could project a student to the LCDs around the room. This classroom could accommodate up to 50 students. Fig. 2 shows the prototypical, state-of-the-art ALC. The traditional classrooms had a row, front-facing student seating with an instructor station located at the front of the classroom. The classroom technology was limited to a PC for the instructor along with a visualizer and an overhead projector. Whiteboards were limited to the front of the classroom.

B. Course Descriptions
CPS 180 is an introductory course in computer programming and algorithm design (i.e., CS1). The course introduces students to the design and development of computer programs in a structured programming language. Core concepts covered include variables, control structures (i.e., selection and repetition), I/O, and methods in the context of the Java programming language and Eclipse-integrated development environment. No prior programming experience is expected of students who enroll in this course. The course content was delivered through short lecture demonstrations, online videos, and an electronic textbook. Students in the course met twice a week, with one meeting for instruction and one meeting for individual laboratory work. Each period was 75 min in length. During the instructional period, students worked extensively in instructor assigned groups to implement algorithmic solutions to provided problems. A typical instructional period consisted of 15 min of instructor-led discussion and exposition followed by 60 min of group activities. The group activities included implementing solutions to problems, debugging programs, and interpreting and explaining code. Periodically during this time, groups were asked to report back with the class and explain their solutions. During the instructional period, students met in the classrooms and made use of any ALC technology available to them. The course content, assessments, and pedagogy remained the same across all sections of the course, regardless of the type of classroom used during the instructional periods (i.e., PALS, ALC, or traditional). CPS 181 follows CPS 180 in sequence and represents a traditional introductory course to data structures and algorithms (i.e., CS2). The course is required for several majors in addition to computer science, such as information technology and computer engineering, and covers multiple concepts. Beginning with object orientation and related topics (such as inheritance, interface, or generics), students continue with software development in Java. Additional topics include, amongst others, algorithmic complexity, recursion, searching, or sorting, as well as introductions of data structures, such as lists, stacks, trees, or queues. Just as in the predecessor course, face-to-face class meetings were typically held twice a week for 75 min. Content delivery made use of active learning approaches in one meeting and the hands-on laboratory assignment work in the second meeting. The active learning meetings were typically structured around brief course management for 5-10 min, followed by two sessions of 1) content review for 15-20 min with 2) subsequent break-out exercises in groups and 3) remerging by having groups present their solutions to exercises to the whole class (with discussion), before concluding the class period. The individual group whiteboards installed in both ALCs and pencil-paper in the traditional classroom were used more often than in CPS 180, as some of the more theoretical topics covered in the course lent themselves toward drawn-out solutions.

A. Design of Study
A quasi-experimental design was used for this study. The design paired the PALS-equipped classroom in head-to-head comparisons with either the prototypical ALC or a traditional classroom. In all, five pairings were made (i.e., two comparing PALS and ALC in CPS 180, one comparing PALS and a traditional classroom in CPS 180, one comparing PALS and ALC in CPS 181, and one comparing PALS and a traditional classroom in CPS 181). The same course materials, assessments, and pedagogy were used in each pairing. All sections of CPS 180 were taught by the same instructor and offered at the same time of day. All sections of CPS 181 were taught by the same instructor (but a different instructor than the one that taught the sections of CPS 180). The sections of CPS 181 were offered at different times in the day due to scheduling limitations. Another notable limitation was that the number of students in the ALC numbered around 45 students while the PALS and traditional classrooms accommodated around 25-30 students. This limitation was due to institutional policies regulating the use of the ALC. Due to scheduling restrictions and the three-year duration of the study, only the PALS and ALC pairing for CPS 180 was repeated.
At the beginning of the semester, students enrolled in sections of the course selected for the study were invited to participate. The invitation was extended by a member of the research team and when the instructor of the course was not present. During the semester, the instructor of the course did not know who had provided consent. The study was approved by the institutional review board at Central Michigan University. Tables I and II summarize the characteristics of student participants and performance by section. For the purpose of reporting and analysis, students enrolled in the PALS sections from each respective course were pooled together. This pooling across sections was also done for ALC sections.

B. Data Analysis
To analyze the data, a series of one-way between-groups analysis of variance was first conducted to explore potential differences between the three class types and two direct  measures that served as the dependent variables: 1) final exam score and 2) overall course score. A similar analysis of variance was also administered on a diagnostic test administered at the beginning of the semester to ensure students were similar in their programming capacities across class types and thereby bolster the validity of the study. Once the potential differences across the three sections were explored, a series of hierarchical linear regressions was utilized to estimate the relationships between additional independent variables of interest (i.e., gender, class year, race, diagnostic pretest, and classroom type) on students' final exam score and the overall course score. Hierarchical regressions enabled an examination of contributions of each of these variables in three blocks: 1) demographics; 2) pretest; and 3) classroom type. Isolating these three blocks allowed for better estimates of the potential unique effects of classroom type (i.e., PALS, ALC, and traditional classroom) on the two dependent variables for CPS180 and CPS181. Tables III-VI list the variables and coding. Gender was dichotomous with males as the referent, class year was categorical with freshmen as the referent, race was dichotomous with White students as the referent, the diagnostic tests were continuous, the classroom type was categorical with ALC as the referent group, and both dependent measures (i.e., final exam score and overall course score) were continuous.

A. CPS180
Potential differences in students' prior knowledge were examined via a diagnostic pretest. Using a one-way betweengroups analysis of variance, no statistically significant differences were found across the three class types at the p < 0.05 level: F(2, 166) = 1.843 and p = 0.585 for CPS180. A oneway between-groups analysis of variance was conducted to explore the impact of class type (i.e., PALS, active learning, and traditional) on final exam score and the overall course score in CPS180. For final exam score in CPS180, there were no statistically significant differences at the p < 0.05 level: F(2, 153) = 1.183 and p = 0.309. For the overall course score in CPS180, there were also no statistically significant differences at the p < 0.05 level: F(2, 152) = 0.578 and p = 0.563.
Next, hierarchical linear regressions were used to examine the effects of demographic variables, a pretest diagnostic measure, and the classroom type on students' final exam score Overall, the final model explained 13.6% of the variance in students' final exam scores. Both the first and second blocks were significant overall, while the final block of classroom type was not statistically significant. White students scored significantly higher on the final exam than students of color. Students who scored higher on the pretest diagnostic exam also scored significantly higher on the final exam. Another hierarchical regression was employed on students' overall course score in CPS180 and the results of the regression model are in Table IV. Overall, the final model explained 17.9% of the variance in students' overall course score in CPS180. Only the first block was statistically significant while the final two blocks were not statistically significant. White students scored significantly higher on the course overall. The pretest diagnostic exam was not a statistically significant predictor on the overall course score. Students in the PALS classroom had significantly higher overall course scores than students in the ALC.

B. CPS181
The diagnostic test for CPS181 yielded no statistically significant differences across the three class types at the p < 0.05 level: F(2, 93) = 2.99 and p = 0.065. Using a one-way between-groups analysis of variance, no statistically significant differences were found across the three class types at the p < 0.05 level for either the final exam or the overall course score in CPS181. For the final exam score, there were no statistically significant differences at the p < 0.05 level: F(2, 91) = 1.490 and p = 0.231. For the overall course score in CPS181, there were also no statistically significant differences at the p < 0.05 level: F(2, 95) = 0.539 and p = 0.585.
Next, hierarchical linear regressions were also used to examine the effects of demographic variables, a pretest diagnostic measure, and the classroom type on students' final exam score and overall course score for CPS181. Table V contains the results of the final block of the regression for students' final exam score. Overall, the final model explained 18.1% of the variance in students' final exam scores. Only the second block was statistically significant while the other two blocks were not statistically significant. No demographic characteristics were statistically significant. The pretest diagnostic exam was a statistically significant predictor of the final exam score. There were no statistically significant differences in classroom type.    of the regression for students' overall course score. Overall, the final model explained 35.6% of the variance in students' final course scores. The first two blocks were statistically significant while the final block was not statistically significant. Students from more advanced class years (e.g., juniors) were more likely to score higher on the final course score. The pretest diagnostic exam was a statistically significant predictor of the overall course score, which explained the largest portion of variance (31.4%). There were no statistically significant differences based on the classroom type.
The hierarchical regressions employed in this study showed permissible variance inflation factors (VIFs) with scores all around 1.0, which indicate no issues with multicollinearity [18]. Missing data were excluded from cases pairwise. Table VII contains a summary of the results for CPS180 and CPS181. Figs. 3 and 4 visualize the significance of the differences found through the hierarchical linear regression analysis for the final exam score and the overall course score, respectively. In Figs. 3 and 4, the size of the circles is proportional to β for each block.

V. DISCUSSION
The focus of the study was the potential effect of classroom type (i.e., PALS, ALC, and traditional) on students' final exam score and overall course score. A statistically significant effect was found for only one measure, which was that students in the PALS classroom in CPS180 scored higher on their overall course grade even when accounting for demographic differences and the pretest measure. There were no other significant effects for classroom type, either on the final exam for either course or the overall course score in CPS181. This lack of additional significant effects suggests that students in the PALS classroom not only performed similarly on their final exam and overall course score compared to the more expensive ALC but also in one instance (i.e., CPS180 overall course score) they performed better. There was only one classroom-type block that showed a significant difference and this difference was on students' overall course score for students in the PALS classroom for CPS180 which accounted for only 2.7% of the overall 17.9%. Results also suggest that students in traditional classrooms performed similarly as well, furthering the argument that classroom type does not seem to matter much on students' final exam score or overall course score.
The reason why the overall model in CPS181 predicting the overall course score explained nearly twice the amount of variance (35.6% variance explained) versus the overall model in CPS180 predicting the overall course score (17.9% variance explained) is unclear. What is clear is that the diagnostic test in CPS181 accounted for 31.4% of variance explained while the diagnostic test in CPS180 did not reach significance. The diagnostic test in CPS181 was much more predictive of the final course score, likely because the questions were more directly aligned to course content. Additionally, CPS181 requires a prerequisite of CPS180, which is an introductory course. CPS181 students have more knowledge about course content than CPS180 students, which attracts many nonmajors. Mean scores were higher on the CSP181 diagnostic test compared to CPS180, too, which could account for the increased differences in total variance explained.
Previous work with the PALS and ALC classrooms investigated students' self-reported perceptions of their learning and experiences (as opposed to performance measures reported in this current study) [19]. In the prior study, students from the PALS-equipped classroom perceived the classroom technology to be more effective on three key measures that were employed on a postsurvey: 1) enhancing their ability to collaborate and work with their peers; 2) enhancing learning overall; and 3) the overall satisfaction with the course. In the prior study, no differences were found in students' perceptions on the benefits of screen sharing specifically or the ability to learn from peers [19]. The perception data did not include traditional classrooms. Considering both the current data on performance and prior research on perceptions, there is corroborating evidence that PALS and ALC classrooms offer similar experiences to students, with some modest evidence to suggest PALS is more beneficial.
The demographic characteristics also reveal noteworthy results. Despite some research that suggests gender differences, the analysis reported here showed no effect for gender in the final model. These results should not be interpreted that gender does not matter in computer science courses; they merely suggest that once all of the variables were considered in the final model, gender did not have a significant effect on the two dependent variables. Other variables (e.g., intent to major in STEM, sense of belonging) may differ. Race, on the other hand, was a significant predictor for both final exam and overall course score in CPS180, but not the more advanced CPS181 course, with students of color performing worse on both measures in CPS180. The class year was only significant for the overall course score in CPS181, with more advanced class years being more predictive of a higher overall course score. As expected, the pretest diagnostic measure was significant even when the classroom type and demographic characteristics were considered, with a positive relationship shown between higher scores on the pretest diagnostic measure and final exam score in both courses and overall course score in CPS181 (there was not a significant relationship with the overall course score in CPS180 and the pretest diagnostic measure). Taken altogether, these results suggest some mixed results, with students in the PALS classroom performing similarly and in one case better than students in the other classroom sections, suggesting that the classroom type has, at most, a modest role in students' performance on the final exam and the overall course score, challenging the notion that costly ALCs bolster student success.
A specific classroom design or supporting technology is not an explicit requirement to employ an active learning pedagogy. Many active learning techniques (e.g., think-pair-share, 1-min paper, and immediate formative feedback) can be applied in a traditional classroom [20]- [23]. The cost of constructing stateof-the-art ALCs has led some to question the cost and if such spaces are needed. A recent study by Greer et al. [24] found that when controlling for pedagogy, the classroom had no significant effect in a CS1 course. Their study investigated the use of peer instruction through small group activities and while their intervention did attempt to leverage the layout of an ALC, the use of collaborative technology was limited.
An important consideration in future research on ALCs is the use of space and technology. Studies need to investigate the specific affordances of the technology and design of ALCs. Some of the skepticism about the reported benefits of ALCs stems from studies that have not controlled for pedagogy (i.e., comparing an active learning pedagogy in an ALC to lecture-based pedagogy in a traditional classroom) [25]- [27]. Likewise, when investigating the technology in an ALC, it is important that the technology is used. One would not expect a significant difference in outcomes if the technology available in the classroom is not used. Particular to programming courses are multiple solutions and implementations for a given problem. The ability to quickly share these solutions with the entire class and mimic discipline-specific practices, such as pair programming and code walkthroughs are, at a minimum, greatly enhanced in ALCs. In general, students have favorable perceptions of ALCs and their collaborative affordances [19].
Apart from the technology, the layout of an ALC can affect instructional choices made by the instructor and students' expectations [28]- [30].
Flexible classroom designs and collaborative technology such as PALS offer institutions a justifiable pathway forward as institutions wrestle with classroom transformations. These options are low cost and comparable to prototypical ALC designs in terms of function and students' performance and perception. Hardware to retrofit an existing computer lab to support eight groups (i.e., eight PALS), three larger classroom displays, instructor station, and whiteboards costs approximately $8200 [17]. This figure will depend largely on the extent of any renovations that are needed for the physical space and do not include labor. Still, the PALS system was designed to leverage the existing classroom configurations and expertise common to engineering and computer science departments so that these additional expenses can be kept to a minimum. Such designs and technology can increase access to active learning environments for students, instructors, and researchers alike.

VI. CONCLUSION
Presented were results of a study that compared an economy ALC, PALS, to a prototypical ALC and a traditional classroom in the context of a CS1 or CS2 course. Analyses of several head-to-head pairings of the PALS-equipped, economy ALC with a prototypical ALC indicated few significant differences in students' final exam score and the overall course score. These results provide additional support for the construction and use of flexible, economical ALCs as they indicate that such spaces perform as well as more established designs. Systems such as PALS support student collaboration and active learning pedagogies but at a fraction of the cost.