iProgVR: Design of a Virtual Reality Environment to Improve Introductory Programming Learning

Currently, there are a plethora of solutions developed to help students learn the basics of programming. However, there is a relative paucity of solutions that cater to problems students face when learning programming that is mainly caused by the abstract nature of programming, misconceptions of programming concepts, and lack of motivation. Hence, in this study, a framework to address the abstract nature of programming and common programming misconceptions is developed. The framework consists of three modules that correspond to each issue, powered by a simulation engine. The first module is developed to address the abstract nature of programming by representing programming concepts with concrete objects in the virtual environment. The second module employs simulation techniques such as interactions and player perspectives to address common programming misconceptions. Lastly, the third module employs elements in the virtual environment to engage students when learning through the system. To evaluate the system, 60 participants were randomly divided into the control group (N = 30) and the experimental group (N = 30). Participants in the control group were taught using a video lecture while participants in the experimental group were taught using the developed VR intervention. Evaluation results gathered quantitatively indicated that the VR intervention was able to significantly increase programming concepts comprehension and address programming misconceptions. Participants also rated the developed VR intervention to be significantly more engaging than the video lecture.

courses with a 7% increase from the year 2017 to 2019 [2]. 23 In the same report by Higher Education Student Statistics 24 (HESA), it is found that the percentage change of computer 25 science courses is the highest among science, technology, 26 engineering, and math subject areas. However, dropout rates 27 The associate editor coordinating the review of this manuscript and approving it for publication was Andrea F. Abate .
for computer science courses are also the highest among other 28 Science, Technology, Engineering, and Mathematics (STEM) 29 courses at 9.8% [3]. 30 Recent studies done on reasons for the high non-31 continuation rates have ranged from negative experiences, 32 low sense of belonging and poor teaching, resulting in low 33 grades [4]. Consequently, Pappas et al. [5] who sampled 34 1050 Norwegian computer science students found that stu-35 dents with higher expectations often fail to fulfil them, which 36 reduces retention. Tan, Ting, and Ling [6] and Medeiros [7] 37 cites reasons such as the abstractness and misconceptions of 38 programming concepts as well as student's lack of intrinsic 39 motivation to be the root cause of failures in introductory 40 North America, and South America) by Bennedsen and 95 Caspersen [9], failure rates of introductory computer pro- 96 gramming courses were found to be at 28%. However, the 97 author notes that this number is subjective across differ-98 ent universities or colleges where elite schools will deem 99 this number high while other schools may find this num-100 ber acceptable. Becker et al. [10] argue that while the 101 contributing factor for the failure rates mentioned above 102 is multifaceted, the root problem encompassing this issue 103 is the difficulty of learning how to program, which ulti-104 mately leads to increased dropout rates. This section aims 105 to highlight some of the issues faced by students when 106 starting an introductory programming course. According 107 to Medeiros et al. [7], the problems faced by students are 108 problem formulation, solution expression, solution execution 109 and evaluation, and behaviour. However, to better situate the 110 study, only the top issue from each category and only issues 111 directly related to building programming knowledge will be 112 highlighted. Hence, the main problems faced by students 113 when learning computer programming are the abstract nature 114 of programming, difficulties with programming syntax, and 115 student behaviour (motivation and engagement). 116 These days, computer programming is mostly taught with 117 high-level programming languages where users only need 118 to deal with variables, arrays, and loops compared to call 119 stacks, registers, and memory addresses. While this improves 120 code readability and usability, users may find programming 121 concepts to be too theoretical, making it harder for some 122 to grasp the fundamentals of programming. This problem 123 is also highlighted in a research by Dunican [11] who pro-124 posed analogies in introducing programming concepts to 125 relate the concept of variables and data types to objects 126 and scenarios in order to aid student's learning. Dasuki and 127 Quaye [12], who interviewed 28 undergraduates from a Nige-128 rian university found that students could not comprehend 129 how programming concepts could solve real-world problems, 130 resulting in disinterestedness towards programming. Accord-131 ing to Giraffa et al. [13], programming concepts are hard to 132 grasp due to the need to imagine abstract terms that do not 133 correspond to real life objects. This issue is also prevalent 134 in literature by Gomes and Mendes [14] and Eltegani and 135 Butgereit [15]. Consequently, despite being the easiest data 136 structure, arrays are often cited as being potentially diffi-137 cult to understand [16], [17], [18]. This ultimately inhibits 138 students from applying the concept when developing pro-139 grams [17]. Hence, it is important to map programming con-140 cepts to real-world contexts to help students understand the 141 fundamentals and what type of problems can be solved with 142 programming. 143 Additionally, the need to simultaneously learn syntax 144 and semantics can be overwhelming [19]. According to 145 Giraffa et al. [13], students often find it difficult to con-146 vert the solution to a problem written in pseudocode into 147 a syntactically correct program. Hence, syntax errors and 148 helping students address them should also be considered 149 as part of the learning process [7]. Koulouri et al. [19] 150 VOLUME 10,2022 showed that the relationship between how an individual 198 perceives the difficulty programming and anxiety levels is The use of virtual environments for teaching is supported by 208 the constructivist theory [30], [31], [32] and the embodied 209 congnition theory [32], [33]. Hence, this section highlights 210 existing techniques that employ virtual reality techniques in 211 developing experiences for use in learning computer pro-212 gramming. Tanielu et al. [34] developed a VR experience 213 called ''OOPVR'' to reduce the abstractness of OOP con-214 cepts with analogies. To increase a student's understanding of 215 objects and classes, OOPVR utilizes a house to depict a class 216 where multiple houses can be built using the corresponding 217 blueprint. Thus, this implies how a class can instantiate 218 multiple objects. Throughout OOPVR, many analogies were 219 implemented to represent programming concepts namely 220 instances, methods, and encapsulation. To determine the 221 effectiveness of the system, the authors analysed results from 222 the pre-and post-surveys that probed 17 students on how con-223 fident they were when it comes to visualizing various OOP 224 concepts. Results revealed that participants were significantly 225 more confident in conceptualizing OOP concepts after the 226 experience. 227 Similarly, Singh [35] proposed a VR experience in which 228 students can learn basic programming concepts by visualiz-229 ing code. Students are presented with a code block in which 230 they will be able to highlight specific lines of code or parts of 231 a line of code to observe how the code is represented in the 232 virtual space. For instance, if the student highlights the code 233 ''int x = 10;'' they will be presented with a cube labelled 234 with the name of the variable (in this case, the variable name 235 corresponds to ''x'') and a sphere to represent the value (in 236 this case, the value corresponds to 10). Students are also 237 required to place the sphere into the cube to represent the 238 assignment of a value to a variable. However, there were no 239 testing done to prove the effectiveness of the VR experience. 240 Vincur et al. [36] combined both VR and game elements 241 with block-based programming to develop ''Cubely'', which 242 is used to teach programming. Cubely contains cubes that 243 represent programming that can be arranged to construct 244 programs. Students are required to build programs from code 245 blocks to control and guide their character to complete chal-246 lenges. According to the study's results, out of 19 partici-247 pants, 18 preferred Cubely because it is easier to use and 248 is more engaging than typical online code bootcamps like 249 Code.org. Bouali et al. [37] developed a VR game called 250 ''Imikode'' to help students familiarise themselves with OOP 251 concepts. This system provides an opportunity for students 252 to create and build virtual worlds using programming. For 253 instance, students can use commands like ''fox = new Fox()'' 254 to create a fox into their virtual world. Despite that, no test-255 ing was conducted to confirm the system's effectiveness on 256 students. 257 Consequently, Chen et al. [38] developed a VR tool that 258 allowed students to create levels in a game to challenge their 259 peers. Before starting the game, the student playing the role 260 of the level creator will write codes in the virtual environment 261 to place robot characters around the environment that will act 262 as obstacles. Students who then play the game are required to 263 acquire hints scattered around the virtual environment while 264 overcoming the obstacles set up by the previous student.

265
In a sense, students will be able to judge the effectiveness 266 of their own code depending on how the system reacts. The 267 authors claim that students of age 9 to 13  algorithms. This is also complemented with the development 329 of an Android app that scans the blocks and compiles them 330 into actual programs that can trigger various events. The tool 331 was tested with students that were between 12 to 13 years old 332 who attended an introductory programming course. Results 333 showed that the tool managed to improve spatial reason-334 ing and programming skills. Consequently, González [46] 335 utilised tangible blocks to identify novice's misconceptions 336 in programming. This is done with the development of an 337 Android application that scans the blocks that are arranged by 338 the students to form algorithms and checks it for any errors. 339 The feasibility of the system was tested with 9 students who 340 have taken an introductory programming course. While the 341 system was well received by the students, the authors claim 342 that the feedback system needs to be improved further as 343 students tend to ignore some of the information displayed and 344 thus missed out on learning.

345
Furthermore, some researchers also employed visualisa-346 tion techniques to help students learn and program better. For 347 instance, Lerner [47] developed an extension for the Visual 348 Studio Code IDE that displays projection boxes which fea-349 tures code visualizations. These visualisations mainly show 350 the state of various data structures in a program during 351 code runtime. The tool was tested with the aid of 10 par-352 ticipants whose experience ranged from medium to expert. Scratch allows users to create online projects with a drag 392 and drop block-based interface [53]. Similarly, Alice allows 393 users to develop and build computer animation with three-394 dimensional models [54]. Blockly, developed by Google is 395 also similar, except it is used in applications where developers 396 are provided the syntax and programming representation to 397 make application building a more seamless process [55].  [58]. According to Seraj et al. [59], when surveyed if 404 blocks or code were preferred for programming, students 405 were found to favor blocks with more girls opting for blocks 406 than boys. Consequently, Milo (similar to Scratch but used for 407 machine learning classes) by Rao et al. [60], found that 90% 408 of the participants agree that block-based programming made 409 understanding concepts easier. However, the drag and drop 410 block-based interface of these programs can pose a program 411 as it generally takes more time to drag blocks than typing [61]. teachers to set up programming-based quests and challenges 426 for their students. By conducting surveys on 30 participants 427 from a high school in Greece, it was found that participants in 428 the experimental group were engaged and motivated through-429 out the experience. However, it was also found that there was 430 no significant difference between the control group and the 431 experimental group in terms of students' performance for a 432 test. Furthermore, Mathrani et al. [64] utilized a game called 433 ''LightBot'', to test the feasibility of the system in regards 434 to teaching programming concepts. In the game, participants 435 are required to guide a robot to solve puzzles using blocks 436 that represent commands identical to code. Evaluation results 437 gathered from 20 participants found that the game success-438 fully introduced programming concepts such as recursion, 439 conditionals, and functions. 440 Wong and Yatim [65] developed a game called 441 ''The Odyssey of Phoenix'' to aid in the learning of 442 Object-Oriented Programming (OOP) concepts. This is 443 accomplished by mapping game processes to their respective 444 concepts. The inheritance concept, for instance, is mapped 445 into the game's crafting element, where resources needed by 446 both the nose and main gear can be shared due to them falling 447 under the gear category. Results gathered from the pre-and 448 post-test of 214 first-year university students were analysed 449 to determine the game's effectiveness. It was revealed that 450 there were substantial differences between pre-and post-test 451 results, implying that GBL is an excellent tool for knowl-452 edge acquisition and learning. Moreover, Oyelere et al. [66] 453 created the ''MobileEdu-puzzle'' game to also teach pro-454 gramming. The game functions by requiring students to 455 arrange disorganized lines of code. It was found that 71% 456 of the 51 students who took part in the study said that they 457 were able to learn programming effectively because of the 458 experience. However, no pre-and post-test were carried out 459 to support this fact. The participants also responded positively 460 when asked if the game managed to motivate them to learn 461 programming. 462 Orehovački and Babic [67] studied the feasibility of 463 ''CodeCombat'' in which participants had to develop and 464 execute Python code to solve quests. Typically, the code 465 is written to move a character from one point to another. 466 Results gathered from 175 participants showed that students 467 had a positive attitude towards the game to learn program-468 ming concepts. Durán et al. [68] developed a game called 469 PLMan where students are required to build controllers with 470 the Prolog programming language. The goal is to develop a 471 controller with a set of basic rules such as ''If a ghost is at your 472 right, move left'' to enable the game character to devour all 473 the dots placed in the maze-like environment. The game was 474 employed as part of a computational logic class for two years 475 during the year 2015 to 2016 with an average of 300 students 476 each term. Results showed that almost 70% of the students 477 had chosen computational logic as their favourite subject.  placed in the control group. Results showed that students 518 who utilized the game had a higher mean score during their 519 midterm exams. It was also found that students found the 520 game to be engaging. Part of that body of knowledge includes both semantic and 538 syntactic comprehension. Semantic knowledge ranges from 539 low-level notions of what subscripted arrays and data types 540 are or how assignment statements work; to more intermediate 541 notions such as summing up all of the elements that are 542 present in an array or developing an algorithm to find the 543 larger of two values; to high-level notions such as recur-544 sion and sorting. On the other hand, syntactic knowledge 545 ranges from familiarising oneself with the valid character 546 sets of a conditional or assignment statement, the format of 547 an iteration, and the names of library functions. To ensure 548 students can master both semantic and syntactic knowledge 549 in tandem, Shneiderman [76] proposed the spiral approach. 550 This approach follows the cognitive model, which works 551 by presenting students with small amounts of semantic and 552 syntactic knowledge at a time. Each time, the new knowledge 553 presented should contain both semantic and syntactic com-554 ponents, should contain minimal addition and be related to 555 the previous knowledge, and presented with meaningful and 556 relevant examples. Existing tools for computer programming 557 education can be grouped into two categories in terms of the 558 approaches employed, the first being the construction-first 559 approach and the second being the comprehension-first 560 approach. The construction-first approach is an iterative pro-561 cess that works by making programmers write code, build, 562 and run the code, observe the output, and then revise the code 563 [77]. On the other hand, the comprehension-first approach 564 prioritises building a mental model of programming seman-565 tics to build intuition which can then be used to build pro-566 grams with fewer semantic-based misconceptions [78].

568
The mapping between similar characteristics of unrelated 569 principles or concepts is known as analogies [79]. In theory, 570 an analogy used during a lesson should be familiar and should 571 have a large degree of correlation between the source and the 572 concept that is targeted. This is because familiar analogies 573 promote learning [80] and increases student engagement. 574 However, it is also important to note that since both the anal-575 ogy and the concept are not identical, a perfect correlation is 576 not possible. The process of teaching and learning computer 577 programming is a tedious task as it requires significant cog-578 nitive, abstraction, logical, and mathematical skills [81]. Due 579 to this, multiple studies have been dedicated to investigating 580 the effects of utilizing analogies in the teaching and learning 581 of computer programming [65] According to Chibaya [83], Since programming revolves 583 around manipulating and managing data, lessons should 584 incorporate mind-maps or visualizations of memory pro-585 cesses from when data is inputted, when data is processing, 586 and when data is outputted. Therefore, it is important to have 587 students visualize the computer memory as a large storage 588 space capable of storing many pieces of data. This means 589 that the addition, management, tracking, and traversing of 590 data elements should be closely thought of as the act of 591 Furthermore, the process of naming and defining a variable 595 is called a declaration, and often, newly declared variables are 596 thought of as empty containers to be filled with a data value.

597
When the need arises, the data value can also be modified. As seen in Table 1, most VR applications for computer pro-   programming misconceptions can be better identified 646 and addressed''. According to Barry Figure 2 shows the system screenshot of how programming 727 concepts are represented in the virtual environment. Drawers 728 represent variables, a row of drawers represents lists while 729 lockers represent 2D arrays. The name of the data structure 730 can also be seen on the monitors above the drawers and 731 lockers to better aid understanding. 732 Figure 3 shows how a challenge is presented and com-733 pleted. According to the spiral approach proposed by Shnei-734 derman [76], small amounts of semantic and syntactic knowl-735 edge should be presented at a time to ensure students can 736 master both semantic and syntactic knowledge in tandem.

737
Hence, the assignment statement presented as the chal-738 lenge (1) will cover the syntactic part while the act of placing 739 the data blocks (2) into the corresponding drawer (3) will 740 cover the semantic part of the spiral approach. For example, 741 if the challenge presented to the user is a variable declaration 742 to assign the value 10 to a variable named total, students are 743 required to grab the data block labelled with the value ''10'' 744 and place it into the drawer labelled with the variable ''total''. 745 This simulates how an assignment statement works in the VR 746 environment.

747
Similarly, lists and 2D arrays are also presented as assign-748 ment statements, requiring students to assign values to the 749 data structures in VR. This is done by placing data blocks 750 in a row of drawers for lists and lockers for 2D-arrays. Fur-751 thermore, the system will use Python code when presenting 752 challenges to students since Python is perceived as easier and 753 more   [23], [24]. Another common 778 misconception highlighted in the developed VR experience 779 are the starting index for both list and 2D arrays, which is 780 commonly regarded as 1 instead of 0 [28], [29]. Hence, this 781 module also addresses the gaps from existing works, where 782 misconceptions have never been addressed in VR (Table 1).

783
Since this is a first-person simulation, students who initially As mentioned, the presence of a narrative context, a reward 795 system, and visual and audio feedback in games can incite 796 engagement and motivation in players [93], [94]. Hence, 797 the system features all the aforementioned elements. Fig-798 ure 5 shows some of the trophies that are part of the reward 799 system in the virtual environment. Furthermore, narrative 800 context is also presented to users during the start of the 801 experience to foster motivation and engagement.

802
Users will first be presented with a narrative context before 803 the first task. Depending on how the user completes the task 804 presented to them, instantaneous feedback will be given in the 805 form of audio cues. If the user completes several challenges 806 in a row correctly, rewards would also be presented. Hence, 807 it can be concluded that the system employs narrative con-808 texts, a reward system and instantaneous feedback to incite 809 learning motivation. Control structures were also included in iProgVR. Control 812 structures like if statements and loops are presented in a 813 similar fashion, where syntax is introduced first, followed by 814 semantics as proposed by Shneiderman [76]. For example, 815 if the code presented to the user is an if statement as seen in 816 Figure 6 (1), students are asked to determine if the value in the 817 variable ''age'' is larger than ''20''. If the value stored in the 818 variable is above 20, it is expected that students will have to 819 grab the ''message block'' labelled with ''you are too old'' 820 in the virtual environment and place it on the output station. 821 By enforcing the action of executing the if-statement in a first-822 person simulation, students can understand the semantics of 823 an if-statement. 824 While there are many types of repetition statements, for-825 loops and nested for-loops were chosen for this study as 826 C. Wee et al.: iProgVR: Design of a Virtual Reality Environment to Improve Introductory Programming Learning  in the for-loop is explained. For instance, the line ''for row 835 in colours'' is explained in such a way that for every ele-836 ment present in the list named colours, store the value in 837 variable ''row''. Then, students will be asked to observe the 838 values stored in variable ''row'' at every iteration. At the 839 first iteration, the value ''blue'' will be seen in variable 840 ''row'', depicted by the drawer labelled ''row''. At the sec-841 ond iteration, the value ''green'' will be seen in variable 842 In the VR environment, game object behaviours are con-882 trolled by scripts. The scripts are added to 3D models such as 883 the ''data blocks'' which defines the behaviour of the object 884 whenever it is interacted with. All scripts are written in C# 885 language and are subsequently imported into the Unity Editor 886 and attached to the 3D objects. VRTK snapzones are used 887 in conjunction with scripts written to dictate the sequence 888 of events. The scripts check to see if the user has correctly 889 carried out the correct actions (i.e. Placing the ''data block'' 890 in the correct spot) and events such as the playing of pos-891 itive audio feedbacks, appearance of trophies and confetti, 892 deactivation of previous challenges, and activation of next 893 subsequent challenges will be set in motion. To create a 894 simulation that is based on a first-person perspective, the 895 tracking origin type of the VR camera rig is set to the floor 896 level to fix the user's position and orientation relative to the 897 floor. A model of human hands was also added with scripts to 898 enable interaction with the Oculus Touch controllers. Addi-899 tionally, control scripts also dictate the function of each but-900 ton on the controllers.

902
As with most studies, alternative methods are considered 903 when coming up with an approach to address the gaps pre-904 sented in this study. Some of the alternative methods include 905 visual programming, tangible tools, and game-based learning 906 experiences. However, these techniques are deemed unsuit-907 able for the aim of this study, which is to solve issues faced by 908 students when learning computer programming. For instance, 909 visual programming makes programming less daunting by 910 allowing users to build programs without the need to worry 911 about syntax [51]. However, since this study aims to address 912 misconceptions and since misconceptions of programming 913 concepts also cause syntax errors [21], the nature of visual 914 programming languages that do not expose users to syntax 915 does not coincide with this aim. For the same reason, visual 916 programming does not meet the requirements of the spiral 917 approach which aims to strengthen both semantic and syn-918 tactical knowledge at the same time [76]. As mentioned, it is 919 important to have students visualize the computer memory 920 as a large storage space capable of storing many pieces of 921 data [83]. Hence, while tangible tools support the construc-922 tivist learning theory [30], [31], it may require extensive effort 923 on the educator's side to prepare elaborate cardboard setups 924 to visualize a computer's memory. However, this is not an 925 issue for pre-developed VR environments that can be shared 926  (1) A for-loop to traverse through a list is presented and explained, (2) The data ''blue'' is seen in variable ''row'' for the first iteration, (3) The print statement is explained, (4) Block to satisfy the print statement, (5) The platform with a snap zone where the block is supposed to go, to simulate displaying outputs to the programmer. and setup with minimal effort. Lastly, while GBL applications 927 for computer programming has also shown promising results, 928 VR environments may be more engaging than non-VR inter-929 ventions. This is also supported by the fact that students 930 generally prefer VR interventions over non-VR interventions 931 in educational contexts [95], [96], [97].

933
The main objective of this research is to determine if iProgVR 934 can help address and mitigate issues faced by students when 935 learning introductory programming. This study will also 936 attempt to compare the effects of the same intervention with 937 and without the presence of VR. The non-VR intervention 938 has all the same aspects of the VR intervention, but players 939 utilize the mouse and the keyboard when carrying out tasks.   Before starting the experiment, participants were briefed 991 on the structure of the study and is made aware that they 992 can drop out of the study if they wish to. The experiment 993 was carried out anonymously, in which participants were not 994 required to reveal their identities. Participants are then given a 995 randomly generated number to decide which group they will 996 be assigned to. Those with odd numbers are assigned to the 997 control group while those with even numbers are assigned to 998 the treatment group. A consent form was also handed out to 999 ensure that their participation was voluntary.

1000
After the consent form was signed, participants (for both 1001 experiments) were first given a pre-experimental survey in 1002 which they were given 15 minutes to complete. The survey 1003 requires participants to answer questions pertaining to their 1004 demographics, their perceptions of their programming skill 1005 and motivation level. The survey also features questions that 1006 were designed specifically to assess their prior understanding 1007 of concepts such as variables, lists, 2D arrays, if statements, 1008 and loops. When filling up the survey forms, participants 1009 were instructed to not discuss their answers with another 1010 party to avoid biasness in their answers.

1011
Once the pre-experimental survey is done, participants 1012 were then asked to complete a tutorial session to familiar-1013 ize themselves with the controls and the surrounding envi-1014 ronment. Particularly, the tutorial session is carried out to 1015 acquaint participants with ways to navigate through the vir-1016 tual environment, ways to carry out gestures such as grabbing 1017 and placing, and the location of important landmarks. These 1018 tasks are applicable to both the VR and non-VR intervention. 1019 For participants assigned to the control group in experiment 1020 1, they will be presented with slides containing materials dis-1021 cussed in iProgVR. Once it is confirmed that the participant 1022 has successfully carried out the tasks in the tutorial session, 1023 the participants were then presented with a set of challenges 1024 (Table 2). 1025 For experiment 1, once the video and all the challenges 1026 in Table 2 were completed by participants in the control 1027 and experimental group respectively, participants were then 1028 asked to complete the NASA-TLX survey and the post-1029 experimental survey. The NASA-TLX survey is used to 1030 access the participant's mental workload while completing 1031 tasks in the VR environment. More details regarding this 1032 will be provided in Section D below. The post-experimental 1033 survey consists of questions that were designed to assess the 1034 participant's comprehension of concepts taught during the 1035 intervention. Other than that, the post-experimental survey 1036 also requires participants to answer questions that assess 1037 their perceived outcome. The assessment questions in the 1038 post-experimental survey feature questions that were similar 1039 to the ones in the pre-experimental questions. This is to 1040 accurately measure if there are any improvements before 1041 and after being presented with iProgVR. Figure 8 shows 1042 the summary of the procedures carried out during 1043 experiment 1.

1044
For experiment 2, participants will then be presented with 1045 another version of the intervention. For instance, if the 1046 participant was first presented with the VR version of the 1047 intervention, they will then be presented with the non-VR 1048     The second section consists of questions specially 1066 designed to assess how well the participants were able to 1067 apply programming knowledge to the real world. The ques-1068 tions were created based on a book by Briggs [101] and Hayes 1069 [102], that specializes in teaching programming to beginners. 1070 For example, participants will be asked to identify if the 1071 statement can either be represented with a variable, list, array 1072 or an if statement.

1073
The third section consists of questions to assess how well 1074 participants can apply their knowledge of for-loops by writing 1075 code to traverse through lists and 2D arrays. The questions 1076 were taken from past year questions from the Programming 1077 Principles course from the university. The fourth section con-1078 sists of questions that are related to common misconceptions. 1079 Specifically, these questions are related to the misconception 1080 that variables can store more than one value, as supported by 1081 by Sleeman et al. [22], Doukakis et al. [23] and a recent study 1082 by Swidan et al. [24] and that the first element of an array is 1083 VOLUME 10, 2022   The experiment took place in the House of Multimodal Evo-1113 lution (HOME) lab at Sunway University, Malaysia for a 1114 period of two weeks. The device used for the experiment 1115 is a laptop running the Windows 10 Enterprise operating 1116 system powered with an Intel Core i7 @ 2.90GHz processor 1117 accompanied with 8GB ram. Along with that, the Oculus 1118 Rift headset and Oculus Touch controllers were used for 1119 participants to experience iProgVR. As for the intervention 1120 without VR, participants used a standard keyboard and mouse 1121 with the same laptop. The paired t-test, the independent t-test, and the ANCOVA 1124 test were used to evaluate data gathered from this study. These 1125 statistical approaches were chosen as the paired t-test is com-1126 monly used to assess participant's pre-and post-scores after 1127 being presented with an intervention [105]. Consequently, the 1128 independent t-test is commonly used to compare the means of 1129 samples from two groups [105]. To analyse student's perfor-1130 mance when presented with the developed VR intervention 1131 as compared to the video lecture, the ANOVA technique is 1132 used as it is often used to analyse the post-test results of 1133 two interventions [105]. All tests employed a confidence 1134 level of 95%.

1135
Having decided to employ t-tests to analyse data, a nor-1136 mality test (Shapiro-Wilk) was carried out to ensure the sam-1137 ples are normally distributed. This is crucial, as this is an 1138 assumption when using t-tests [106]. As shown in Table 4, the 1139 test generated a non-significant result (more than.05) which 1140 indicates normality [106].

1143
As mentioned, a paired-samples t-test was carried out to eval-1144 uate the impact of iProgVR on participant's scores on the pre-1145 and post-assessment. As shown in Table 5  A paired-samples t-test was also carried out to evaluate 1163 the impact of the video lecture on participant's scores on the 1164 pre-and post-assessment. As shown in Table 6, it was found 1165 that the responses to the pre-and post-assessment questions 1166 were statistically significant for variables (M = −53.83, 1167     the ANCOVA test can be used to analyse student's post test 1202 scores. As shown in Table 7, there was a significant difference 1203 between both groups on their post-test scores F = 16.248, p 1204 <.05. This indicates that iProgVR improved student's scores 1205 significantly more than the video lecture.  An independent-samples t-test was carried out to compare 1209 the post-assessment scores for female and male participants. 1210 As shown in Table 8 To assess participant's perceived outcomes, the mean was 1222 calculated from the Likert Scale based survey questions. 1223 As seen from Table 9, participants in the control group (M = 1224 5.90, SD = 1.062) were more likely to agree that they prefer 1225 to have analogies presented to them visually in the virtual 1226 environment than spoken about verbally compared to those 1227 in the experimental group (M = 5.17, SD = 1.577). It was 1228 also found that this is significant, at t(58) = −2.112, p =< 1229 .05. Participants in both the experimental (M = 5.70, SD = 1230 1.343) and control group (M = 5.93, SD = 1.202) agree that 1231 the iProgVR and the video lecture can be good supplements 1232 VOLUME 10, 2022

1253
The difference in scores were found to be significant at,  An independent samples t-test was carried out to determine 1309 if there are significant differences between the performance 1310 of participants who were presented with the VR and non-VR 1311    presented. This suggests that by representing abstract con-1325 cepts as concrete objects and by allowing students to identify 1326 and rectify misconceptions, programming knowledge can be 1327 improved. Despite so, performance scores were rather low at 1328 the end of the evaluation. This is expected, given the short 1329 amount of time allowed to learn four major programming 1330 concepts as well as programming misconceptions.

1331
The first research question posed was ''How will the repre-1332 sentation of programming concepts as concrete objects in the 1333 virtual environment aid in students' understanding of abstract 1334 concepts?'' Along with this research question, the hypothesis 1335 proposed states that ''Abstract programming concepts are 1336 easier to understand if it is represented as concrete objects in 1337 the virtual environment.'' The hypothesis is supported by the 1338 evaluation results, whereby students' post-intervention scores 1339 for the variables, arrays, if statements, and loops were signif-1340 icantly better than the pre-intervention scores. This suggests 1341 that by representing abstract concepts as concrete objects 1342 in the virtual environment, programming knowledge can be 1343 improved. These results build on existing evidence of the 1344 ability of VR to help students effectively visualize abstract 1345 programming concepts as seen in [34].

1346
The second research question posed was ''How will the 1347 ability to identify and address common misconceptions of 1348 programming concepts by interacting with the virtual envi-1349 ronment aid in reducing misconceptions?'' Along with this 1350 research question, the hypothesis proposed states that ''By 1351 allowing students to interact with the virtual environment, 1352 programming misconceptions can be better identified and 1353 addressed.'' As seen in Table 5, the hypothesis is supported 1354 by the evaluation results, whereby students' post-intervention 1355 scores for the variable, list, and 2D array-related miscon-1356 ception were significantly better than the pre-intervention 1357 scores. This suggests that by allowing students to identify and 1358 address programming misconceptions in the virtual environ-1359 ment, misconceptions can be mitigated. These results build 1360 on existing evidence of the being able to improve knowl-1361 edge acquisition when allowed to interact with one's envi-1362 ronment [107], [155], [156]. 1363 Furthermore, as seen in Table 6, comparisons in perfor-1364 mance scores between the VR intervention (iProgVR) and the 1365 video lecture was significant. This indicates that participants 1366 who were presented with the iProgVR scored significantly 1367 better than those who were presented with the video lecture. 1368 As online classes become more popular due to the Covid-19 1369 pandemic, this finding can be useful for future applications 1370 of remote learning. This finding is synonymous with find-1371 ings from existing studies that utilised VR and traditional 1372 teaching methods in comparing student's assessment scores 1373 [39], [108]. However, as seen in Comparing post-assessment scores between beginners and 1389 those without any programming experience, performance 1390 scores were not significant as seen in Table 7. This shows 1391 that the system is suitable for those with or without program- can help supplement lessons [112], [113].

1422
When asked if they prefer learning through VR experi-1423 ences, those presented with the VR intervention were more 1424 likely to agree compared to those presented with the video 1425 lecture. While previous research has found that students 1426 generally prefer VR over a non-VR setting when learning 1427 [95], this result provides a new insight into the relationship 1428 between exposure and interest to learn with VR. These results 1429 also build on existing evidence of students generally prefer-1430 ring VR over a non-VR setting in educational contexts [95], 1431 [96], [97]. Surprisingly, when asked if they preferred learn-1432 ing through video lectures, participants in both groups also 1433 agreed. This indicates that students are mostly interested to 1434 experience new teaching forms. Results from this study can 1435 be used as a basis for improving remote learning experiences 1436 as this way of learning becomes more popular. In terms 1437 of perceived confidence levels, participants (both groups) 1438 agree that the experience increased their confidence in terms 1439 of programming knowledge. While existing evidence exists 1440 on the ability of VR to improve confidence levels [114], 1441 this indicates that while VR interventions can increase one's 1442 perceived confidence on a particular subject matter, video 1443 lectures are no different.

1444
The third research question posed was, ''Will the devel-1445 oped intervention increase student engagement when learning 1446 computer programming?'' Along with this research question, 1447 the hypothesis proposed states that ''The developed inter-1448 vention can help with student motivation and engagement 1449 when learning computer programming.'' As seen in Table 8  Lastly, when asked if the concepts presented was easy to 1458 understand, participants in both groups agreed. This indicates 1459 that they found that the concepts were presented in a simple 1460 and easy way. Interestingly, this also means that the mere 1461 existence of VR does not make content less complicated. 1462 Which indicates that if lessons are presented appropriately, 1463 knowledge can be absorbed by students efficiently with or 1464 without the presence of VR. This is further supported by 1465 participants (both groups) agreeing that both VR and video 1466 lectures can enrich the learning process. Despite so, par-1467 ticipants who were presented with the iProgVR had better 1468 assessment scores than participants who learnt through video 1469 lectures. This is a weird discrepancy as participants equally 1470 agree that both experiences made learning easy and enriched 1471 learning. One explanation that can be given is that while 1472 the video lecture did a good job in presenting the concepts, 1473 student's may have been bored by the monotonous nature of 1474 the medium, thereby decreasing retention rates. This can be 1475 corroborated by comments given by one participant who said: 1476 ''The presentation can be more engaging. The slide itself was 1477 not eye-catching and I believe there a more concise methods 1478 used by YouTube tutors which I find to be more helpful in 1479 terms of ever-lasting knowledge. Besides that, this lecture 1480 was helpful.'' 1481 C. WORKLOAD 1482 As seen in Table 9, participants (both groups) expressed that 1483 the mental demand needed to complete both interventions 1484 was high. This indicates that both interventions required 1485 participants to use rather high levels of thinking, calcu-1486 lating, searching, and remembering when completing chal-1487 lenges presented to them. This is expected as participants 1488 had to learn similar concepts in both interventions. These 1489 results are synonymous with findings found in an article 1490 by Berkman et al. [115] who reported that comparisons of 1491 mental demand scores in VR and without VR for a puzzle 1492 game were not significant. However, there are also conflicting findings that report significantly higher mental demand scores for the VR version of the experience compared to the 1495 non-VR version of the same experience [116], [117], [118].
Participants also rated the physical demand to be somewhat high for the VR intervention (iProgVR). This could be due 1498 to the need for physical activities such as turning one's head 1499 and body to teleport around the VR environment. The need 1500 to grasp virtual objects with the use of controllers could also 1501 be the reason. Surprisingly, participants also rated physical 1502 demand to be somewhat high for the video lecture as well.   These results are synonymous with findings found in an 1549 article by Berkman et al. [115] and Rao et al. [116]. 1550 Participants were also asked about their frustration level, 1551 particularly how stressed, annoyed, and discouraged did they 1552 feel during the tasks. It was found that participants in both the 1553 experimental and control group rated their frustration level 1554 to be high. This could also be due to the need to learn so 1555 many new concepts, one after the other in a short period of 1556 time. These results are synonymous with findings found in an 1557 article by Rao et al. [116]. However, there are also conflicting 1558 findings that report significantly higher frustration levels for 1559 the VR version [117] while another study reported report 1560 significantly higher frustration levels for the non-VR version 1561 of iProgVR [115].

1562
The inconsistencies in results from this study and existing 1563 works when it comes to the workload of VR games and 1564 interventions indicate that workload is largely dependent on 1565 the tasks one needs to carry out in the virtual environment. 1566 For instance, the workload may differ across VR and non-1567 VR platforms for a puzzle game [115], a shoot training 1568 simulator [116], [117], and a wheelchair simulator [118]. 1569 Regardless, the workload results discussed above can be a 1570 good benchmark for future research when determining work-1571 loads for VR and video lectures in the context of teaching and 1572 learning computer programming. As seen above, feedbacks left by participants are use-1590 ful for future renditions of the intervention. For instance, 1591 the current work can be expanded to feature collaborative 1592 elements. Thus, more tests should be done to find out if 1593 performance can be increased when participants are allowed 1594 to work and interact with one another in the virtual environ-1595 ment. Participants also expressed interest for more immersive 1596 environments, particularly one where they can explore by 1597 walking around. Related to this, is the desire to move freely, 1598 without the restriction of wires that are connected to the 1599 VR headset. While this can be achieved with wireless VR 1600 headsets like the Oculus Quest, an expensive contraption 1601 such as the KAT Walk C would need to be purchased to 1602 simulate walking around the virtual environment. Participants also provided suggestions to change the colour scheme of 1604 the virtual environment into less drabby and brighter colours. 1605 One participant also expressed their interest for a workshop 1606 that feature iProgVR. This can certainly be done once the 1607 pandemic situation improves. Lastly, some participants also 1608 wished to have more time to process the concepts that were 1609 presented to them in the virtual environment. As mentioned,  senting the loss as a virtual object in the computer-generated 1657 space, it is hypothesised that grief can be more effectively 1658 tended to. 1659 With the increasing popularization of VR technology, it is 1660 not a surprise if virtual environments are adapted more 1661 in future classrooms. Results from this study may also be 1662 applied to provide a foundation to make VR classrooms a 1663 reality.

1665
This paper outlined several issues related to learning intro-1666 ductory programming. Firstly, is the need to address the 1667 abstractness of programming concepts where a framework 1668 is proposed to represent programming concepts as concrete 1669 objects in a virtual environment. Secondly, is the need to 1670 address misconceptions of programming concepts where a 1671 simulation design for misconceptions of programming con-1672 cepts is proposed. Thirdly, is the need to incite intrinsic 1673 motivation when learning programming. To handle this issue, 1674 a simulation technique to incite learning motivation and a 1675 simulation module is also proposed. Results gathered from 1676 testing indicated that the intervention successfully improved 1677 student's knowledge acquisition as seen from the significant 1678 improvements in the pre-to post-test scores. Furthermore, 1679 results also show that students were motivated and engaged 1680 throughout the whole experience. This indicates that unlike 1681 existing tools for teaching computer programming (as dis-1682 cussed in Section II), the intervention developed can be used 1683 as a one stop solution to mitigate issues commonly faced by 1684 students when learning programming. Results also showed 1685 that there is no difference when it comes to workload and 1686 perceived outcomes (except engagement) when comparing 1687 iProgVR and the video lecture. However, it was found that 1688 students were more engaged and scored higher assessment 1689 scores in the experimental group. This means that iProgVR is 1690 a feasible system. Comparisons were also done with the VR 1691 and non-VR version of the same intervention to address the 1692 lack of such studies, specifically in the context of computer 1693 programming education. It was found that the presence of 1694 VR does not affect knowledge attainment. However, since the 1695 nature of experiment 2 is at its preliminary stage, these results 1696 should not be used as a basis. As mentioned, since most 1697 works in this field are limited to conference papers [90], this 1698 means that not much comprehensive testing has been done in 1699 existing works. To address this, this study has provided new 1700 knowledge by conducting more thorough test that includes 1701 workload, pre-and post-assessments of both experimental 1702 and control groups, and perceived outcomes that are measur-1703 ing more than student engagement and motivation (as seen in 1704 Table 1). Additionally, since there is also a relative paucity of 1705 studies that compared the VR and non-VR version of the same 1706 intervention in the context of teaching and learning computer 1707 programming, this study would also aim to carry out such 1708 comparisons albeit in a preliminary manner, to provide a 1709 foundation for future studies.

1710
In the future, further work can be done to complement 1711 the system. Particularly, in terms of providing more support 1712 for different programming languages. This would allow the system to be used by a wider range of audience so as not limit the learning of multiple programming languages. Furbased misconceptions should also be implemented to address 1718 common programming misconceptions. While this study is meant to benefit instructors who are already aware of the 1720 benefits of VR and is looking for VR experiences that can do more than engage students when learning programming, 1722 future work can also be done to assess iProgVR to other active He is also the Founder of the Human-Machine Col-2192 laboration Research Centre (HUMAC) that aims 2193 to be nation's main technology hub and to demon-2194 strate its commitment to sustainable development. 2195 He has spent 20 years working in data networks 2196 (IT), telecommunications, computer networking, haptics, manufacturing 2197 industries, and electronic control system in machinery. His research project 2198 on DHVE is meant to have multi-sensory feedback data, such as voice, 2199 audio, and force over the fixed and wireless network. He is a principal 2200 investigator for a few projects that are supported by the Ministry of Higher 2201 Education (MOHE), ERGS, MCMC, Lancaster University, U.K., PPRN, 2202 industrial partners, and Sunway University Internal Grant, respectively. His 2203 current research interests include haptic over the distributed virtual envi-2204 ronment (DVE) in HCI environment, haptics and odor sensing, drones, 2205 odor sensing/tracking, tele-haptics, tele-robotics, materials sensing, assistive 2206 technology for limited vision and hearing impaired, and AR&VR.