Robotics vs. Game-Console-Based Platforms to Learn Computer Architecture

The ﬁelds of robotics and game consoles offer an interesting and broad range of lab platforms with appropriate characteristics for teaching Computer Architecture concepts. This work analyzes the impact of one approach based on game consoles and another one based on robotics from a triple dimension: student motivation, acquired knowledge, and perception of the employed platform. The study has been carried out on a sample of 96 students using the Arduino-based robot and 75 students using the Nintendo-DS console. A mixed methodology is employed encompassing quantitative and qualitative approaches. Five instruments are used to measure the three aforementioned dimensions. Results show that despite both platforms performing similarly in the three considered dimensions (student motivation, acquired knowledge, and perception of the employed platform), the robotics platform does it slightly better than game console, based on the obtained average scores for the considered instruments. Despite this outperforming, motivation and perception decrease for the students using the robotics platform as result of some identiﬁed constraint. This suggests that changes are required in the organization of the lab sessions to promote teamwork skills and to overcome the lack of simulators to remove the obstacles hinting motivation and performance. However, a clear correlation between motivation and perception and acquired knowledge has not been identiﬁed on computer architecture. Implications of affordances and constraints of both platforms, types of activities, and their impact on results have been discussed


I. INTRODUCTION
The Association for Computing Machinery (ACM) and IEEE Computer Society Joint Task Group on Computer Engineering Curricula consider the Computer Architecture and Organization area part of the body of knowledge for the curricula of Electrical Engineering, Computer Engineering, and Computer Science [1]. Despite its importance, this area has traditionally suffered from poor student performance [2]. The fact that most Computer Engineering or Computer Science students are more interested in software aspects than in those related to hardware [3] and those with hardware interests tend to choose Electrical and Electronic Engineering degrees is one of the reasons explaining the lack of The associate editor coordinating the review of this manuscript and approving it for publication was Tomás F. Pena . engagement and the under performance in this area in the different degrees.
The implementation of the European Higher Education Area brought along new teaching methodologies and evaluation methods that are not performing as well as expected [4] what is increasing the pressure for new approaches that help improving students' motivation and performance.
Teaching Computer Architecture is therefore a challenging task. The role that the practical platform plays in the teaching process is essential, and many different approaches have been proposed to this end [5], [6] being a recurrent discussion whether simulators or virtual laboratories should be adopted instead of real platforms [7].
Any tool used for this purpose should be suitable to develop the called ''computational thinking'', defined as the skill needed to solve problems effectively and VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ efficiently in different contexts [8]. This skill is not necessarily linked to computers as is a daily life ability that every citizen needs [9], but is essential to understand and communicate computational concepts effectively [10]- [12] and programming adequately, a reason why it is a basic part of almost all engineering programs curricula [13]. Abstraction is particularly relevant in this sense to reach computational thinking at a high level [14], [15]. It is noticeable that students that follow courses of Computer Architecture are usually in the first course of the degrees, what implies they are frequently unexperienced with these fundamental skills. Many of them are not enough familiarized with suitable forms of notation, necessary to develop mental models [16], together with a lack of programming concepts and language syntax, what may lead to demotivation, hindering learning [13], so the pedagogical approaches used in teaching should be aimed at fill in these gaps. Robots and computer games reach this objective.
Traditional approaches for teaching Computer Architecture frequently leads to a poor performance of students, being far from a constructivist approach [17] in which the knowledge is actively constructed by the learner [18], [19]. In this vein, Papert and Harel in [20] coined the term constructionism, a derivative or constructivism, based on learning is much more meaningful when students create tangible or shareable artifacts, mobilizing their knowledge and skills [21]. Robots [22] and computer games [23], [24] are good examples of engaging artifacts to discover knowledge with educational interest in Computing Education.
The benefits that Nintendo DS brings as a lab platform for teaching Computer Architecture concepts have been demonstrated [25], [26] and experienced by the University of Castilla-La Mancha, pioneer in adopting this platform during the academic year 2007/2008, as a pedagogical resource. Despite being a great pedagogical asset, Nintendo DS is, at the moment, an obsolete gaming platform. Finding replacement is becoming more and more challenging. For that reason, the academic staff involved in teaching this course faced the dilemma of whether Nintendo DS should be replaced by another game console or change the approach to the currently hot-topic issue of educational robotics. The study presented here was conducted to collect evidences and make an informed decision on what approach should be implemented next: a game-console based or a robotics one.
Both robotics and game-console based platforms have demonstrated positive impact on student motivation [27]- [31]. Additionally, their hardware features are very interesting, in terms of provided resources and constrained complexity, for the purpose of teaching Computer Architecture. These two approaches therefore satisfy the Computer Engineering and Computer Science students demand of having attractive and real platforms for experiencing with Computer Architecture concepts [32]. However, to the best of our knowledge, there is a lack of works quantitatively assessing the impact that both approaches have on learning Computer Architecture. To fill this gap, this paper presents two specific platforms representing these two approaches: a fit-for-purpose Arduino-based robot and the Nintendo-DS console. Appropriate hands-on sessions have been designed to cover the content of the course Computer Structure and learning outcomes have been gathered, analyzed and compared from the perspective of the student motivation, knowledge, and perception of the employed platform.

II. EDUCATIONAL ASPECTS OF GAME-BASED AND ROBOT TEACHING APPROACHES
Despite the importance that practical training has on the learning process to give teaching a constructivist approach and promote learning, students very often find this experience compromised by the lack of appropriate equipment [33]. In this reality, the use of software-based simulators has become a common practice for undergraduate courses, especially in science and engineering disciplines. The advent of advanced computer graphics, augmented and virtual reality are leading to sophisticated virtual laboratories that provide a close-to-real experience overcoming the inconveniences associated to real laboratories (equipment failure and maintenance, associated cost, limited number, etc.) and allowing the students to focus on the most important aspects. However, these approaches are still at a very early stage and their use is commonly reserved for supporting initial steps. Still, the use of computer-based simulators has an important negative impact on the student motivation [34] who rather use traditional real equipment for more in-depth hands-on experiences [35].
It is therefore evident that a trade-off must be found between the positive impact of using real platforms and the overwhelming details that they might entail. This is what the work in [36] has referred to as the professor's dilemma. Students demand real platforms that provide them with a realistic picture of what is on the market, whereas on the other hand, market products are specifically designed to maximize market penetration and company profits, which make them, most of the time, inappropriate for learning purposes. An additional obstacle that must be faced when looking for a real platform for teaching Computer Architecture is the fact that students are polarized regarding their interest about software and hardware aspects. Finding a platform that fits these two polarized groups is not easy because those keener on hardware-related aspects might found as a too limited platform one that might be perceived as too cumbersome for those other more interested in software aspects [26].The solution to this dilemma consists in employing a platform with a fast learning curve but with enough challenging resources to meet the expectation of those more experienced students.
In this sense, the use of game-based approaches has different pedagogical advantages. Previous research found that a lack of initial affinity for playing computer games was not a barrier to enjoy the scientific experience of game construction [37]. These authors also highlighted gamebased construction learning as a suitable approach to develop higher-order thinking and Computer Science abstraction skills, becoming an enjoyable approach for teaching Computer Science Education, even if they remark as a constraint the sophisticated level of programming skills knowledge needed.
Different approaches can be found in the literature that resort to game-based approaches for teaching Computer Architecture. The work in [38] describes how mobile educational games can be used to overcome the limitations of traditional classroom approaches, focusing in providing a fun, interactive and motivating way of discovering and experiencing with computer architecture concepts. Despite the strengths that this proposal has on how contents are delivered, limitations are identified as this is not appropriate for acquiring low-level programming skills. The work in [39] proposes a game using 16 cards, each of which has assembly instructions written in them, as a first approach for getting students familiarized with the course contents. This is a very interesting approach, but it is only intended to cover the first sessions of the course. Finally, the work in [25] focuses on the role that game-console-based project can play in learning the Input/Output subsystems. The work of Larraza-Mendiluze et al is totally in line with the approach presented here, nonetheless, the approach presented here is not only limited to the Input/Otuput subsystem, but it also covers the rest of subsystems comprising a computer.
An alternative tool for teaching Computer Architecture, which is also very appealing to students, is the use of robotics, and more precisely the so-called educational robotics. This approach has recently gained attention [40] as more evidences are being obtained demonstrating the positive results in learning [41], [42]. Due to the versatility of robot platforms [30], a wide range of applications can be pursued, turning this platform into a unique tool to target a wide audience, from primary and secondary school students up to higher education [43]. The use of robots in education has benefits as the development of thinking and social skills [44], are of high interest in all educational stages. Robots are also a powerful tool in the teaching and learning process due to their interactivity and flexibility [30], [45], and the wide variety of enjoyable hands-on experiences they provide [46]. Platforms like Arduino [47], [48], Lego Mindstorms NXT [49], [50], or Raspberry Pi [51], and programming environments such as Logo [52], [53] or Scratch [13], [54] are some of the most popular tools. These platforms and programming environments have, as their leitmotif, to provide simple, intuitive, and easy-to-use tools. The so-called maker movement or making,Giannakos2017 is a new concept that has been recently coined to describe those activities that put the focus on the process of learning by constructing. Despite being a relatively new concept it is based in the well-known philosophy of learning by making,Papert1991 and founded in a constructivist (and constructionist) approach of the teaching and learning proces [44].
The use of robotics in primary and secondary education is an emerging trend mainly due to the vast amount of information available online, thanks to the open-source community that is generally supporting some of these projects, as well as the attention that STEM (Science, Technology, Engineering, and Mathematics) subjects are receiving as enablers for skills such as the computational thinking [56]. Higher education is not unaware of this trend but educational robotics is being employed as a complimentary tool [30] because students, at this level, should have already acquired these skills. In this sense, the use of robotics has been mainly employed to promote motivation and student engagement [30], [31], [45], [57]. They have also pedagogical benefits in learning [58], as they are based on problem-solving and active thinking [59] especially in the way lessons have been designed in the present research. This process of problem-solving activates cognitive skills like representing (the external representation of a problem is transformed into an internal mental model), planning, executing and evaluating [60], besides the development of metacognitive thinking skills needed to solve problems [61].
Additionally, their hardware features are very interesting, in terms of provided resources and constrained complexity, for the purpose of teaching Computer Architecture. However, to the best of the authors' knowledge, there is no similar approach that employs educational robotics, as a lab platform, for teaching computer architecture concepts. These two approaches therefore satisfy the Computer Engineering and Computer Science students demand of having attractive and real platforms for experiencing with Computer Architecture concepts [32]. This work is also motivated by the lack of works quantitatively assessing the impact that both approaches have on learning Computer Architecture. To fill this gap, this paper presents two specific platforms representing these two approaches: a fit-for-purpose Arduino-based robot and the Nintendo-DS console. Appropriate hands-on sessions have been designed to cover the content of the course Computer Structure and learning outcomes have been gathered, analyzed and compared from the perspective of the student motivation, knowledge, and perception of the employed platform.

III. STUDY CONTEXT
Computer Structure is one of the courses that comprise the Computer Architecture area. This is taught as part of the Computer Engineering Degree program in the University of Castilla-La Mancha, Spain. This course is intended to lead students into basic computer architecture concepts such as the memory system, the instruction set architecture or the input/output system. To this end, the course is organized in 6 European Credit Transfer System (ECTS) 4 of which are devoted to theory and 2 for labs. These 6 ECTS will be taught for 14 weeks, having 3 sessions per week of 90 minutes each. One of these three weekly sessions will take place in the lab and the remaining two will be devoted to theoretical concepts. The syllabus is organized in five lessons as stated in Table 1. The first two sessions of the hands-on experiences will be intended to an introduction to the C programming language. The last 4 sessions will be devoted to work on the course project in which the different contents of the course will be put into practice.
This course is taught during the second semester of the first year. Three more additional courses are taught as part of the computer architecture core knowledge, namely: Computer Technology (first semester of first year), Computer Organization (first semester of second year), and Computer Architecture (first semester of third year). The course is taught in Spanish but also in English for those students going for the bilingual mention. The Computer Structure course has been selected for this experiment mainly because it has traditionally suffered from the poorest performance, out of all courses comprising the Computer Architecture area. The failure rate of this course yearly reaches around 50%.
The learning objectives of this course (like the rest of the degree) have been certified by the National Agency for Quality Assessment and Evaluation and Accreditation from Spain and are listed below: • Objective 1: To understand the principles of computer architecture.
• Objective 2: To know the organization of the CPU, identify the functional units, and explain their role in the execution of the instructions.
• Objective 3: To know the organization of the Input/Output subsystem and its interface with the CPU.
• Objective 4: To relate the evolution of CPU architecture and instruction sets. To identify the differences between the CISC and RISC philosophies.
• Objective 5: To program a computer at a low level. • Objective 6: To learn, through practice, the structure and programming of a basic computer.
To address these objectives the course is organized in five lessons, as described in Table 2, addressed through lectures and hands-on experience.
The first lesson consists in an introduction to the structure of a computer. This lesson will offer a macroscopic vision of what will be studied, in more detail, in the later lessons and which makes up the so-called digital computer. The classical architecture proposed by von Neumann serves to briefly introduce the fundamental components of a computer, as well as the basic structures used in its interconnection. As this is an introductory lesson, the corresponding sessions of the laboratory were dedicated to familiarizing the student with the Arduino Zero platform or the Nintendo DS and their respective development environments.
The Nintendo-DS group faces this lesson from the perspective of the a von Neumann architecture although, as processors themselves, the ARM7 implements a von Neumann architecture whereas the ARM9 implements a Harvard one. The ARM9 has two Tighly Coupled Memory (TCM), one for data and one for instructions. The DTCM, with a size of 16 KiB, is where the stack is located. For this lesson, students are prompted to display several backgrounds. The size of the background and the representation mode employed (framebuffer, rotoscale, or tiled) determines whether the background can be properly displayed or not (run out of memory). This exercise therefore makes students aware of the different type of variables, different memories and different types or architectures. Finally, the characteristic parameters of a computer are also explored through the analysis of the platform itself.
The Arduino-based robot group, on the other hand, works with a Harvard architecture. This is however a hybrid Harvard architecture meaning having separeted buses for data and instructions only under specific situations. However, there is no need, at this stage to discuss whether this is a pure or hybrid Harvard architecture. Students experiment with a basic ''Hello, World!'' program in which the string (Hello, World!) is provided in different ways, each of them with different implications. This introduces students into the use of different type of variables (static or automatic) and different type of memories (one for data and one for instructions). Finally, the characteristic parameters of a computer are also explored through the analysis of the platform itself.
This lesson is therefore addressed through different graphical representation modes in Nintendo DS and, in Arduino, through different versions of a basic ''Hello, World!''. First sessions are therefore more attractive, in terms of achieved results (different graphics represented in the console), for the Nintendo DS groups. The Arduino robot-based group, on the contrary, does not experienced any interaction with the robot itself. The serial console and debugger (to explore the memory) are the only elements employed during this session.
The second lesson studies the memory system. The memory system of the Nintendo DS is probably its most valuable asset for teaching Computer Architecture. The video memory (VRAM) plays an essential role for the graphics programming which requires a deep understanding of memory organization. Framebuffer, rotoscale and tiled are the representation modes supported by Nintendo DS. Each of these involves totally different forms of dealing with the memory for graphics representation purposes. For example, the framebuffer mode maps screens to specific memory regions. Writing to this memory region has a direct effect on what it is being displayed in the screen. On the other hand, the tiled mode considers the screen as a matrix of tiles, each of which is a bitmap of 8×8 pixels. This matrix is provided as references to a memory where the tiles are stored. Students in this group are introduced to the graphics programming modes and different exercises were proposed to this end. Understanding direct and indirect addressing modes are essential for an effective use of the different graphic modes. Regarding the Arduino-based robot, unlike generalpurpose computers, the Arduino Zero (as a typical microcontroller) is designed to be dedicated to a single task, which is why a Harvard architecture is so convenient. This group explores the memory system resorting to the Application Binary Interface (ABI) as well as the different regions that determine the state of running program. In this sense, the status of a running program is determined, at any particular time instant, by the content of the memory and registers and, more specifically, by the content of each of the sections into which the memory is divided (text, data, .bss, stack and heap). In the case of the Arduino Zero, these sections will be mapped into specific physical memory spaces (Flash and SRAM). Students are offered a series of simple programs to explore these sections. Although the use of the debugger (KDbg [62]) may be useful, it is generally preferred that students use pointers and print memory addresses. In this manner, they also become familiar with the use of C pointers. In addition, the ABI and, more specifically, the machine's alignment rules and endianess will be used to explore the platform memory system.
Lesson 3 studies the set of instructions that a processor can execute, known as Instruction Set Architecture (ISA). The ISA is closely related to the ABI rules studied in the preceding lesson. A fundamental question for an ISA is to know the different addressing modes that are available. During this lesson different modes will be studied and we will find out which of them are available for the ARM architecture and how they are implemented in its ISA.
The machine and assembly language is experienced through the use of the KDbg debugger, for both groups (Nintendo DS and Arduino-based robot). KDbg is just a front end of the GDB debugger with support for remote debugging. It offers an easy-to-use tool for students to explore the assembly language associated to C code. From a pedagogical perspective, the use of a debugger offers a very powerful mechanism for students to analyze in detail the Application Binary Interface (ABI), the role it plays, and how it is connected to the Instruction Set Architecture (ISA). Students, for example, are prompted to verify that the binary code generated by the compiler complies the ABI rules when it comes to procedure calls. To this end, students resort to the use of a debugger like KDbg to explore not only the assembly code in a friendly manner, but also the content of the processor registers. The KDbg debugger lists the assembly code associated to a C statement in the source code. Students can therefore get familiar with the assembly code without being overwhelmed with all the details involved in an assembly file.
It is important to note that writing an entire program in assembly code has an associated complexity that is beyond the scope of the course. However, having a general understanding of the basics of assembly programming is useful for code optimization and debugging tasks. For this reason, students regardless of the employed platform are provided with a series of code fragments that they will have to analyze, using the functionality offered by the KDbg debugger.
The Arduino-based robot group explores the ISA of the Cortex M0+ in more detail. Students are prompted to write VOLUME 8, 2020 assembly sentences using the ''asm'' function to introduce delays (with the ''nop'' operation) or to move data into registers, verifying aftewards the content of the loaded register with the debugger.
Lesson four is not experienced first hand with any of the platforms. On the contrary the content of this lesson is approached from the point of view of knowing the different elements that make up the datapath (ALU operations, registers, stack, buses, etc.) and, therefore, the activities developed along the previous sessions have already introduced all these concepts. This lesson, as such, does not involve concrete hands-on experiences but we consider them part of the previous activities.
The fifth lesson deals with the input/output system, which along with the processor and memory, is one of the essential parts of a computer. The input/output system will, in essence, enable the communication of the computer with the external world (people and other devices). Both platforms have an interesting list of peripheral devices. For example, the Nintendo DS has two 2D graphics engines, a 3D graphics engine, a touchscreen, 16 sound channels, stereo speakers and microphone, timers, etc., while the Arduino-based robot has a list of sensors and actuators such as ultrasound, infrared, light, buzzer, servo or mini-servo.
Nintendo DS employs a memory-mapped input/output system, meaning that handling the peripheral devices of the Nintendo DS basically consists in reading and writing from certain memory positions. The input/output registers are therefore mapped into specific memory positions that students are prompted to identify. To this end, the library header files are shown as the resource from which this type of information is provided.
Finally, the three input/output modes (programmed, interrupt and DMA) are presented through different examples. In this sense, the Nintendo DS is an excellent hardware to experiment with these three modes. Students are requested to write a simple game in which the three modes are employed. The programmed mode is experienced through the use of keys, the interrupt through the use of timers and graphics scrolls whereas the DMA is employed for loading graphics from memory into the screens. The benefits of using DMA is clearly experienced when loading backgrounds in the Nintendo DS screens. The use of DMA leads to instant loading of graphics whereas the use of the processor (not the DMA) leads to a gradual load of the graphics that it is easily perceived by the human eye.
The Arduino-based robot group has a first contact with each of the sensors and actuators of the platform. After the isolated experimentation, students are requested to integrate as many of them as possible into a line-follower robot. Advanced functionalities are considered to extend the line follower. For example, the robot can be guided by following a source of light or it can play different sound frequencies according to certain events. Students are prompted to extend the functionality as desired, employing as many peripheral as possible. Students soon realize that dealing with different devices call for more advanced management mechanisms. This leads them to consider an interrupted input/output, as an alternative to the programmed one, when tasks involving several devices cannot be attended simultaneously. Table 3 summarizes the main affordances and constraints of the two platforms employed in this experiment: the Arduino-based robot and the Nintendo DS. These affordances and constraints are analyzed in the light of the content of the lessons comprising the course.
In the overall, these lessons are covered during the fourteen lab sessions available all over the course. The three first lessons are employed to introduce students into the C programming language. There is one lab session per week plus two in-class lectures.

IV. METHODOLOGY
The main aim of this study was to evaluate the convenience, from a pedagogical and motivational perspective, of using either robotics or game-consoles based platforms for teaching Computer Architecture. Provided the lack of studies evaluating and comparing these two strategies, this paper pursues the following specific research goals: • To compare both strategies in terms of student motivation, acquired knowledge, and perceived usefulness.
• To identify the pedagogical strengths and weaknesses of both approaches.

B. STUDY DESIGN
The methodology used is based on the mixed paradigm, since the research has used qualitative and quantitative instruments to collect data. Two of the instruments were applied both at the beginning and at the end of the course (pre-test and post-test) and the platform perception instruments (Arduino and Nintendo DS) and the inquiry instrument that were only applied at the end of the course.

C. INSTRUMENTS
The data collection process has been automated using forms. One form has been created for each of the instruments that comprise the study. The Microsoft Forms platform has been employed for being this platform integrated into the corporative network. This has simplified the process of student identification as well as data collection and processing stage. Students were informed, before accessing the questionnaires, about the aim of the study. Only the professors directly involved in the study have has access to personal data (name, gender, age, etc.) which have been anonymized for their statistical analysis by assigning an ID number to each subject. Students were also told that the anonymization process would take place prior to the statistical analysis of the data. This was intended to minimize bias in the responses of students who believed that the outcome of their responses might affect their final grade. Additionally, students were assured before the start of a questionnaire that none of the answers provided would affect their final grade. VOLUME 8, 2020 The ordinal nature of the variables in the Likert scale instruments determined the use of polychorical correlation matrix for validation purposes. Thus, the reliability of the Likert scales instruments have been tested through the ordinal Chronbach's alpha. All the Likert scale instruments were also tested in terms of content and construct validity.
Motivation and acquired knowledge have been considered as it seems that the first one has an impact in the second [63], specially when computer technologies are involved [64], [65].

1) INSTRUMENT 1: STUDENT MOTIVATION
The work in [48] analyses the convenience of state-of-theart instruments for measuring the students' attitude towards introductory programming. The same conclusions apply to our study since learning low-level programming is one of the main competences of the Computer Structure course. We have therefore put the focus on low-level programming.
Among the considered instruments the TAM model [66] is considered to most appropriate one because, apart from being successfully used in [48], it was originally envisioned to estimate the acceptance of innovative information systems. Similarly to the work in [48] a translation was conducted precising low-level programming when a reference to programming was made.
The main constructs of the TAM model are: the perception of usefulness, the perception of ease of use and the behavioral intention to use. This model was therefore employed to inspired this instrument that measures the student motivation towards the low-level programming skill about to be acquired during the course. The questionnaire, listed in Table 4 is comprised of 10 questions which use a Likert scale with 5 possible answers, as known: Totally agree (5), agree (4), neither agree nor disagree (3), disagree (2), totally disagree (1).
Despite being based on a validated questionnaire [48], the validity and reliability parameters have been calculated for our study. The content validity was tested by a panel of ten experts on Computer Architecture, and they agreed that the items were suitable for the research, well written and easy to understand. The construct validity was tested by an Exploratory Factor Analysis (EFA). The Kaiser-Meyer-Olkin sample suitability test (0.758) and the Bartlett's sphericity test (p<0.01) confirmed the pertinence of an EFA. The EFA revealed a structure in which items are grouped into three factors, according to the theoretical construct considered for its development: Factor 1: the perception of usefulness (items 1, 2, 3, 4), Factor 2: the perception of ease (items 5, 6, 7), and Factor 3: the behavioral intention to use (items 8,9,10). The reliability of the scale was tested by the ordinal Chronbach's alpha. The results indicated a very good reliability of the scale [67] as a whole (0.78) and by factors (0.77 for factor 1, 0.72 for factor 2 and 0.76 for factor 3).
The validity of this instrument was carried out by a panel of 10 experts (8 professors of the Computer Architecture and Technology area, 1 undergraduate student, and 1 postgraduate student). Following the first revision, the proposed amendments were undertook and a second version of the questionnaire was resubmitted to this panel of expert who finally consider it valid. This version was applied through a pilot test of 10 students of the Degree in Computer Engineering, proving that there were no doubts in the understanding of any of the items. Due to the dychotomical character of these variables, the reliability of the instrument was calculated using the ordinal Chronbach's alpha. The result (0.67) indicated a good internal consistency of the instrument and makes it suitable for the research.
Additionally, this questionnaire also collects information about students' previous experience. To this end four questions will formulated: 1) Do you have previous experience in programming microcontrollers?; 2) If so, which ones?; 3) Do you have previous experience in computer programming?; 4) If so, what languages have you used?

3) INSTRUMENT 3: STUDENT PERCEPTION OF THE ARDUINO-BASED ROBOT PLATFORM
Students' perception of the computing platforms used in teaching provides important information for assessing the teaching practice [68]. This instrument is intended to evaluate the student perception of the Arduino-based robot platform. The questionnaire proposed in [48] inspires the current instrument to evaluate the use of the Arduino board in the context of low-level programming.
The instrument is comprised of 10 items which use a Likert scale with 5 possible answers, as known: Totally agree (5), agree (4), neither agree nor disagree (3), disagree (2), totally disagree (1). This questionnaire, listed in Table 6, has been inspired in [48].  This instrument is based in a previous one, already validated, but it was necessary to validate it in the context of the present research. Firstly, the content validity was tested by a panel of experts on Computers Architecture, who agreed in the suitability of the items for the research purposes, written expression and ease of understanding. Secondly, the construct validity was tested by an EFA. The pertinence of carrying out an EFA was confirmed by the Kaiser-Meyer-Olkin sample suitability test (0.874) and the Bartlett sphericity test (p<0.01) results. The EFA revealed a construct structure of three factors, according to the theoretical construct considered for its development: Factor 1: Perceived usefulness of the Arduino board (items 1, 2, 3, 4), Factor 2: Perceived Ease of Use of the Arduino board (items 5, 6, 7), and Factor 3 (items 8,9,10). The reliability was tested by the ordinal Chronbach's alpha, due to the ordinal nature of the variables. The results indicated an excellent reliability of the scale [67] as a whole (0.9) and by factors (0.88 for factor 1, 0.85 for factor 2 and 0.90 for factor 3).

4) INSTRUMENT 4: STUDENT PERCEPTION OF THE NINTENDO-DS PLATFORM
This instrument is intended to evaluate the student perception of the Nintendo-DS platform. The questionnaire proposed in [48] has been adapted to refer to Nintendo DS instead of the Arduino board, as in its original version. This questionnaire is listed in Table 7.
The instrument was validated in terms of validity and reliability for this research case study. The content validity was tested, as in the previous cases, by a panel of ten experts on computers education. They agreed that all the items were pertinent and well written, easy to understand and to reply. The construct validity was tested with an EFA. The Kaiser-Meyer-Olkin sample suitability test (0.841) and the Bartlett sphericity test (p<0.01) confirmed the pertinence of carrying out an EFA. The results confirmed that the items were grouped in three factors, according to the theoretical construct considered in the design: Factor 1: Perceived usefulness of the Nintendo DS platform (items 1, 2,3, 4), Factor 2: Perceived Ease of Use of the Nintendo DS platform (items 5, 6, 7), and Factor 3: Perceived Enjoyment when using the Nintendo DS platform (items 8,9,10). The reliability of the scale was confirmed by the ordinal Chronbach's alpha, due to the ordinal nature of the variables. The results indicated a very good reliability of the scale [67] as a whole (0.81) and by VOLUME 8, 2020  factors (0.75 for factor 1, 0.70 for factor 2 and 0.74 for factor 3).

5) INSTRUMENT 5: QUALITATIVE EVALUATION OF THE EMPLOYED PLATFORM
This instrument consists of the following open question: ''Briefly tell us your impressions, assessment, or suggestions for the future about the course and the hands-on experience with the Arduino-based robot/Nintendo DS''. This instrument was applied to a panel of 10 experts composed of undergraduate and postgraduate students and professors from the area of Computer Architecture and Technology. The instrument was applied in a pilot test with 10 students.

D. DATA ANALYSIS PROCEDURE
The SPSS v. 23, R and Factor software were used for data analysis. The Kolmogorov-Smirnov test (as the sample is larger than 50 subjects) was applied to the variables corresponding to the different instruments based on the Likert scale. Since these variables do not meet the normality criterion (p<0.05), non-parametric statistics have been used [69]: Mann Withney's U test for independent samples and Wilcoxon test for related samples. The analysis was complemented with the effect size with r, due to the nonnormality of the variables, but also the mean and standard deviation as indicative descriptors.

A. STUDENT MOTIVATION
When the inter-group analysis is done using the Wilcoxon test for related samples (see Table 8), in the Arduino group it turns out that in 4 of the 10 items there are statistically significant differences between the pre and post test (items 3,5,9,10). In these four items they score more in the pretest than in the post-test (they are demotivated in these four aspects). However, in the Nintendo-DS group, statistically significant differences appear in 3 of the 10 items (1, 2, 6), and in all three, this group scores more in the post test than in the pre test.
The results listed in Table 9 of Mann Withney's U test (comparing independent samples) reveal that in the pre test both cohorts had statistically non-significant differences in scores on 6 of the 10 items (items 1, 2, 3, 4, 6, 7). In the first four and 7, the students in the Arduino cohort showed greater motivation, and in 6, less than those in Nintendo-DS cohort. In the post test, no statistically significant differences were found in any of the items of this instrument.
The analysis of the results measuring the student motivation towards the use of low-level programming shows a decrease in the perception of the usefulness for those using the Arduino-based robot. This decrease affects both to the use of low-level programming during their studies and professional career. Particularly interesting is the decrease experienced in item 5, that regards with the easiness of learning lowlevel programming. This same item does not suffer significant differences for the Nintendo DS group, although the post test score was lower, as an average score, than that obtained for the Arduino group (2.99 for the Arduino group versus 2.96 for the Nintendo DS group). For items 9 and 10, that regards with the perceived utility of low-level programming for the future professional career, a similar situation occurs as described for item 5: from a pre test with higher values in the Arduino group there is a statistically significant decrease despite the fact that the final scores are still higher (on average) than those obtained by the Nintendo DS group, in which there are no statistically significant changes between the pre test and post test.
Note that r gets low values (see Table 10) for both Nintendo DS and Arduino and therefore the effect of the intervention is not relevant in terms of motivation.

B. ACQUIRED KNOWLEDGE
The results of Mann Withney's U test show that there are no statistically significant differences between the two cohorts in either the pre test (p = 0.153) or the post test (p = 0.319). Table 11 summarizes the learning outcomes comparing the score in the pre and post test, with scores in the range of 0 to 10. Although the differences found between groups are not statistically significant, in the pre test, the Nintendo DS group scores slightly better (median = 7.0, mean = 7.16, SD = 1.83) than the Arduino group (median = 7.0, mean = 6.84, SD = 1.80). In the post test, it is the Arduino group that scores better (median = 9.0, mean = 8.06, SD = 1.61) than the Nintendo DS group (median = 8.0, mean = 7.85, SD = 1.60).
However, when the inter-group analysis is done for related samples using the Wilcoxon test, it is evident that there are significant differences between the pre test and post test for both the Arduino group (p=0.000) and the Nintendo DS group (p=0.009). To deepen these results, the size of the effect has been calculated using the r coefficient for nonnormal distribution variables [69]. The result shows a greater effect of the intervention in the Arduino group (0,32) than in the Nintendo DS group (0,58).
A comparison has been also carried out considering independent questions (see Table 12). Results of the Chi-Square test show that initially both cohorts showed non-significant    differences in 6 of the 10 questions that composed the questionnaire, and after the interventions, non-significant differences were found in all the items. The questions with lower scores after the interventions were those related to micro controllers, memory, and instructions.
It is worth noting that students' previous experience in both cohorts is very similar. 28% of the students in the Arduino-based robot group have previous experience in programming microcontrollers (all of them reported having used Arduino or Arduino-compatible board) and 19% have previous experience with the C programming language, whereas for the Nintendo DS cohort, 22% have previous experience with microcontrollers like Arduino or similar and 17% have previously programmed in C.

C. STUDENT PERCEPTION OF ARDUINO PLATFORM
This instrument was applied to the Arduino group at the end of the intervention. A descriptive statistical analysis has therefore been carried out. The results are shown in Table 13. The item that obtained the lowest score regards the perception of the utility of knowing low-level programming during their studies (item 3). On the contrary, the one that obtained the highest score regards the perceived utility of knowing lowlevel programming for their future jobs (item 9).

D. STUDENT PERCEPTION OF NINTENDO-DS LABS
This instrument was applied to the Nintendo DS group at the end of the intervention. A descriptive statistical analysis has therefore been carried out. The results are shown in Table 14.     The item that obtained the lowest score regards the perceived simplicity of learning low-level programming (item 5). The one that obtained the highest score regards the perceived utility of knowing low-level programming for their future jobs (item 9).

E. QUALITATIVE EVALUATION OF THE EMPLOYED PLATFORM
This tool was applied to both groups. Out of the 171 answers, 54 were discarded because they were left blank or did not know the answer. 67 answers of the cohort 1 (Arduino group) and 50 of the cohort 2 (Nintendo-DS group) were analyzed. Table 15 shows the results of the categorized answers, organized by cohorts.

F. CORRELATION BETWEEN ACQUIRED KNOWLEDGE, MOTIVATION, AND PERCEPTION OF THE PLATFORMS
When motivation and knowledge are correlated, inconclusively results are found. Before the intervention, nonsignificant correlations have been found between these two variables considering all the sample (Spearman Rho = 0.075, p = 0.330), the Arduino group (Spearman Rho = 0.121, p = 0.240) and the NDS group (Spearman Rho = 0.082, p = 0.484) separately. Nevertheless, after the intervention, significant correlations have been found for all the sample (Spearman Rho = 0.191, p = 0.012) and Arduino group (Spearman Rho = 0.247, p = 0.015), and non-significant correlation for NDS group (Spearman Rho = 0.115, p = 0.326). Thus, it seems that motivation is not determinant to acquired knowledge, but further research is needed to deep in these results.
Besides, non-significant correlations have been found between the perception of the Arduino platform and the acquired knowledge (Spearman Rho = 0.168, p = 0.102), and in the perception of Nintendo DS platform and the acquired knowledge (Spearman Rho = 0.052, p = 0.661) of the respective groups after the intervention.

VI. DISCUSSION
This study compares two approaches to teach Computer Architecture: Arduino, a robot-based approach widely used from primary to higher education, and Nintendo DS, a game-based one. The results show that, despite the a priori advantages of Arduino as a more engageable tool [30] and pedagogical affordances related to easiness of low-level programming and machine and assembly language, the differences among both approaches at the end of the interventions are not conclusive in terms of motivation and learning outcomes. These results are discussed in this section regarding the different types of learning activities, pedagogical affordances and constraints, and attitudes developed by the students in both didactic sequences.
Students' motivation towards programming was measured as previous literature revealed it has an impact on motivation to learn and acquired knowledge [63], especially when robots are used for teaching [30]. It is stunning how motivation in the Arduino group decreases after the intervention, on the contrary to what happens with the Nintendo DS group, even when the Arduino group was expected to be more motivated according to previous literature [30], [63] given the characteristics of the tool that facilitates programming and provides more interaction with hardware (calibration of sensors, servo, robot pieces, etc.). The Arduino group was indeed initially more motivated than the Nintendo DS one, probably because the use of robots is usually more motivating to learn programming [63]. But at the end, differences in motivation between both groups were not significant. Particularly, the Nintendo DS cohort increased their motivation towards low-level programming, and they perceive that after the intervention it was easier than initially expected. This can be explained by the fact that the game-development process has a positive impact on motivation and harder work when solving problems [13], [70]. Further explanations might be found in different aspects. The work in [71] points out that the use of robots does not necessarily increases the relevance, confidence or satisfaction when programming. The results of our study support these ones, as other aspects play a role in the teaching and learning process that should not be overlooked. Thus, regarding the differences in the low-level programming teaching approaches, the Nintendo DS group resorts to the use of different graphic representation modes (frame-buffer, rotoscale, and tiles), while Arduino group employs simple programs to be analyzed with a remote debugger. The use of graphic modes has an inherent complexity, especially for those facing for the first time computer graphics. On the other hand, first sessions of the Arduino group spin around the use of simple programs which will softly guide them through the different lesson contents. This implies that the robot-approach entails less complexity in terms of low-level programming competences than that for console-approach, taking into account that the Nintendo DS activities need a basic understanding of how graphics and memory work. This different level of complexity in the first sessions is a pedagogical advantage of Arduino over Nintendo DS. However, the fact that first activities of the Arduino group are carried out on the Arduino board (without involving the robot until lesson 5), may lead students to perceive these programming activities as decoupled from the rest of the sessions (implementation of the robot functionality). This is a constraint that directly impact on the learning process for the Memorylesson concepts. The Nintendo DS cohort performs better that the Arduino one on questions regarding the memory system design. All these issues have determined that the Nintendo DS group increases it motivation towards the usefulness of low-level programming, counterbalancing initial motivation differences among groups.
The different characteristics of Nintendo DS and Arduino that determine the learning activities can also explain the differences on perception of both platforms, after the lessons, regarding the easiness to program. Arduino cohort has a better perception of the platform than the Nintendo DS one. Arduino, through its Arduino IDE (programming environment), provides a simple and easy-to-use environment to program the board, what has turned it into a widely used platform from primary to higher education. This is not the case for the Nintendo DS having to resort to commandline tools which might overwhelmed students unfamiliar with the GNU/Linux like systems. This result is of interest as it points out to Arduino, over Nintendo DS, for those students unfamiliar with GNU/Linux systems. This requires further research as this question was not initially part of the research and there were no statistically significant differences between the two cohorts in terms of acquired knowledge either for the pre-test or the post-test.
The differences found between both approaches can also be explained by the way teaching activities have been organized. Arduino activities needed from robots, and were carried out during the lab sessions, in groups. Nintendo DS activities based on simulators and real consoles, enable students to work at home, in an individual manner, preventing some teamwork issues from occurring, even when both approaches were based on teamwork. The higher load of teamwork is a strong point of the Arduino learning activities, as it helps students to develop social skills and work in an environment that is closer to their future work environment and very valuable skills in Computer Science [72]. However, teamwork has inherent difficulties to deal with, as those derived from the conflicts that may emerge among peers, as the answers of the students to the open question show. This result is especially valuable because, even when previous literature has focused on the benefits of developing teamwork skills of game-based approaches [73], for robotics further research was claimed [74], [75], and this work contributes to give light to this issue. Nevertheless, aspects specifically derived from teamwork have not been specifically assessed, further research is needed to evaluate conflicts and their reasons, and the way they assume responsibilities, and how these issues affect the results of the interventions. Nonetheless, the work in [76] recommends working in groups of two people rather than in larger groups because in this way the benefits of pair programming are enhanced and the exchange of leadership roles is facilitated.
The design of both teaching approaches was also different in terms of the introduction of the console and the robot. The Nintendo DS group combine both the console and its emulator counterpart from the first session, what enables a seamless transit to the real platform for less experienced students [35]. On the other hand, in the Arduino group, the robot was not introduced until the Input/output lesson (last lesson of the syllabus). Initially, it was unclear whether this later use of the platform could have an impact on acquired knowledge. The Nintendo DS group started from lower scores than the Arduino one in the knowledge questionnaire (pre test), but in the post test there were no statistically significant differences among both cohorts, being scores around 8 out of 10 in both cases. These high scores support the evidences found by other authors [25], [30], [32], [37], [44], [48], [74], [77] about the convenience of both strategies, robotics and gamebased approaches, for teaching Computer Architecture. That implies that both approaches are equally suitable in terms of learning outcomes, and valid to teach Computer Architecture with similar learning results. Besides, these learning outcomes seem not to be correlated to motivation, in contrast to previous research that suggest that motivation has an impact on acquired knowledge [63], especially when computer technologies are involved [64], [65]. In this case, both approaches seem to homogenize the final perception, independently of the employed platform. It is also outstanding how more than a third of the Nintendo DS cohort regrets the possibility of new students not having the opportunity of programming a game for the console, what aim at the acceptance that the platform has despite its obsolescence.
Despite the lack of statistically significant differences both in the acquired knowledge and at the level of the single questions comprising the questionnaire, it can be noticed that there are three questions (questions number 4, 5 and 10) whose results are under the 75% although over 60%. This open new and future lines of research to explore how these issues could be improved (new activities or improved ones) with the considered platforms.

VII. CONCLUSION
The results of this study show the suitability of the robot-based (Arduino) and the game-based (Nintendo DS) approaches to teach Computer Architecture in terms of learning outcomes. Nevertheless, the different affordances and constraints had also an impact on results due to their differences from a pedagogical perspective.
Arduino is revealed as a powerful approach to teach lowlevel programming to poorly experienced students due to the facilities that the tool presents. It is also useful to teach input/output concepts since it gives students the opportunities to program different behavior in the robot (line follower, obstacle avoidance, light follower, etc.) and deal with external conditions that affect the robot sensors or other peripheral devices. On the contrary, Nintendo DS is especially interesting because of the different graphic modes it support and the different peripheral devices (keys, touch screen, timers, etc.). The design and implementation of a game, including the representation of graphics and its logic involve dealing with different type of memories and input/output techniques. Students can experience the three types of input/output techniques: pooled to manage the console keys, interrupt to use timers, and DMA to display different backgrounds (copying backgrounds to the video memory).
However, besides these pedagogical approaches, teachers must consider other constraints and affordances in their teaching, as the results of this research reveal. The impact that external conditions have on the robot sensors means that it is necessary to calibrate them whenever conditions change. This process is time-consuming, especially in the beginning when students are not familiar with the process. It is therefore desirable to make robots available in addition to the lab hours to minimize frustration that might demotivating them. This is not necessary for game-based approaches, as a simulator is available. This enables students to work individually at home. In this sense, teamwork should not be overlooked, as even when it resemble a real working context, it may trigger conflicts causing delays that could eventually lead to increase the time needed to complete the proposed tasks. It seems also advisable to provide students with the opportunity to work with robots platform from the beginning of the course, avoiding this way the perception of lab sessions being decoupled from input/output lesson. This might be responsible for a low perception of the usefulness of low-level programming. Regarding game-based approaches, teachers should pay attention at initial knowledge requirements on low-level programming to prevent students from stucking from the beginning.
Despite the obsolescence of Nintendo DS, the results of this work are interesting for teachers aimed to design teaching plans based on other game platforms, and of course, to design their robot-based lessons, even using other platforms than Arduino. Further research in other contexts as in vocational training computer programs and in Secondary Education are proposed as future lines of research. Finally, as this is a case study and results cannot be extrapolated, further research is needed to deep in these results.