The Impact of Problem-Based Serious Games on Learning Motivation

Students’ motivation in computer graphic courses is a challenge for educators. Computer courses entail abstract notions and algorithmic features, and these difficulties affect the students’ motivation. A problem-based learning serious game framework named Immersivio was designed. The researchers designed a quantitative study with a between subject design and implemented a treatment course using the problem-based serious game. The game was designed based on the principles of problem-based learning and serious games, i.e., higher order thinking, collaborative learning, and cognitive thinking and information visualization theory. A non-random sampling procedure (purposive sampling) was used in this quasi-experimental study and the participants formed to 2 groups, the experimental group (n = 24) and the control group (n = 26). MANOVA test results indicated that the experimental group participants who had experienced the Immersivio game were more motivated to learn computer graphics in terms of extrinsic motivation, intrinsic motivation, interest, attainment, cost, identification with academics, self-efficacy, and instrumentality. This study has implications for basic and advanced computer graphic lecturers and educators in this field.


I. INTRODUCTION
Three-dimensional representation of geometric data in computer courses is known as computer graphics. Recent studies that have been conducted on learning computer graphics, such as the one conducted by Suselo, Wünsche and Luxton-Reilly revealed that the approaches to learning computer graphics should be novel [1]. They assume that the traditional forms of teaching computer graphics, such as the text books, are very challenging for the students. This requires according attention to recent forms of education and to merge them with learning computer graphics. In addition, Nishino, Sueyasu, Kagawa, and Utsumiya assert that the current learning aids such as 2D images in textbooks for understanding 3D theories and program codes for explaining specific algorithms make learning computer graphics difficult for the learners [2]. They further explain that learning computer graphics requires understanding 2D and 3D environment aspects and acquiring skills such as modelling and programming for 2D and 3D computational environments.
It seems that making use of cognitive and metacognitive psychology can enhance learning of computer The associate editor coordinating the review of this manuscript and approving it for publication was Songwen Pei . graphics courses. Kim, Park, and Baek point to the positive features of cognitive educational psychology which can not only motivate the learners, but also affect their learning [3]. These features are accompanied by the use of technology which can further facilitate learning. This idea formed the basis of Serious Games (SG) in education [4]. Serious Games (SG), as defined by Junaeti, Sutarno, and Nurmalasari is a game that has a purpose beyond pure entertainment, i.e., education [5]. Jung, Yi, and Choi also posit that SG has two main dimensions, i.e., education and entertainment [6]. They indicate that both aspects should be taken into account in designing SG, and one should not be ignored in favour of the other.
In addition, research has shown that motivation is among the most significant aspects of education in any field to the extent that Donnelly speculates that what the learners need for learning is only motivation [7]. Traditional forms of education, which are lecture-based and less interactive, usually do not motivate the learners to learn [8]. It also justifies why SG should be motivating, as the level of engagement with the learning content is high. In comparison to SGs, lecture-based instruction lacks interaction with the learning content. As a result, the learners in lecture-based instruction are passive recipients of knowledge, whereas, through SG, the VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ learners play the most active role in learning. Lack of interaction and being passive learners reduces motivation and causes boredom [9]. Motivation is usually a student-centred concept and gains more significance in learner-centred approaches to learning. SG is also a learner-centred approach to learning. Thus, it is of prime significance to know how a particular form of SG can motivate the learners.

II. PROBLEM
Borrego, Fernández, Blanes, and Robles believe that students who take computer courses have a lack of motivation to the extent that it affects their overall score and performance [10]. Similarly, Papastergiou found motivation as a problem for learning computer science in Greek [11]. Also, and in order to justify the need for this study, the researcher conducted a preliminary study and examined the motivation of 39 Malaysian undergraduate students at Asia Pacific University (APU) where the researcher intended to conduct the study. It was observed that the students' motivation for learning computer graphics is low. To be more specific, it was found that the students are not willing to take challenges to learn as the course content, as the course does not arouse their curiosity to learn.
The students also noted that they do not feel confident that they learn the concepts in the 3D computer course, especially the complicated ones. Many scholars support the idea that computer games can motivate the learners to learn computer programs and computer graphics [11], [12]. These studies also support games as a possibility to solve the learning problem. For example, Papastergiou reported that the motivational appeal of a computer game for learning computer memory concepts is higher than traditional forms of education [11].
However, not all the games that have been designed so far for learning computer programming and computer graphics been successful. Romero report that a common problem of SGs is that they do not consider both the pedagogical and technical perspectives [13]. Elsewhere, Hirumi, Appelman, Rieber and Van Eck posit that problem-solving and collaboration should be included in the games designed for learning any form of computer programming [14]; however, these aspects are usually missing. Among the papers which have reviewed game-based education, the one by Walker & Shelton provides useful insights [15]. They claim that the existing educational games are similar to the works in Problem-Based Learning (PBL). This claim is made based on conceptual and empirical literature. However formal efforts to investigate the similarities between educational games and problem-based learning warrants more research. Thus, this issue should be subject to research. Therefore, what is the effect of a problem-based SG (Immersivio) on learning motivation of computer graphics?

A. LUDOLOGY
According to Caillois Avedon and Sutton-Smith, games are not considered only as entertainment [16], [17]. They are the focus of educational and philosophical debates. Such debates form the basis of science of Ludology. Ludology discusses the effects of games on education through meticulous designs and functions [18]. Although ludology is a term seen across many disciplines, its definition varies. Bragge and Storgards posit that this diversity is the result of intricacies attributed to each discipline [19]; especially as the objective of ludology in different disciplines varies. The discipline of computer science is a core discipline in ludology [20]. This claim is based on two significant reasons. Firstly, many games are designed using the computer technology [21]. Secondly, the human-computer interaction is among the most significant areas in gaming [22]. The main research trends based on these two assumptions include user research [23] and psychophysiology communities [20]. Thus, the focus of ludology is not solely on analysing games or creating a design terminology [24], [25]. One of the main debates in ludology is the need for a methodology with a psychological perspective [26]. Elsewhere, Lindley and Sennersten argue that psychological concepts such as cognition, schema, and motivation should be added be considered in this methodology [26]. Therefore, the researchers in this study made use of a problem-based SG which considers cognitive thinking as a core concept which effects the motivation. As motivation and cognition are concepts relevant to the users as players who experience the game, the next section of this paper is discussed the users' experience.

B. USER EXPERIENCE
There has been extensive research on gamification and user' experience [27]- [29]. Most of these studies have used an experimental design to study the users' experience. Nacke and Lindley who reviewed this study believe that there is a lack of well-accepted and common theoretical definition of the constructs that are often studied in user experience and gamification [30]. They also indicate that the reason game play industry has hindrances in its growth is a lack of common understand of such concepts. However, it can be inferred that the game industry is based on the users' experience [20]. It makes user experience one of the significant concepts in this study.

C. PROBLEM-BASED LEARNING THEORY
According to Norman and Schmidt, this theory has its roots in the experiential learning theory which emphasizes on handson experiences while learning [31]. The theory advocates use of the learners' cognitive skills such as problem-solving while learning; believing that through cognitive learning and in line with principles of higher-order thinking, learning can begin with evaluation of the learning problem and ends it understanding. Selecting PBL as the main educational aspect of this study is based on the above firm reason. The researcher has reviewed the previous games designed for learning computer graphics. The shortcoming of these games can be well covered by using a problem-based design. For example, TUGS (The Universal Graphics System) designed by Clevenger, Chaddock, and Bendig [32] lacks a psychological basis for defining learning, whereas PBL is underpinned in cognitive learning and considers psychology of learning one of the main aspects of learning. A game based on OpenGL designed by Woo, Neider, Davis and Shreiner [33], lacks an educational underpinning and looks at learning computer graphics from a computer-based perspective solely. However, PBL is an educational approach which can suit learning of computer graphics when implemented in a SG. Finally, collaboration which is a fundamental aspect of successful learning has not been in neither of the games mentioned above. Even recent games such as the CodeRunner GL designed by Wünsche [34] has not defined collaboration in the game and in the learning process. These pieces of evidence can prove that a problem-based SG should be more successful than previous games designed for learning computer graphics.

D. LEARNING MOTIVATION
Learning motivation is not a new concept. There are evidences of studying this variable for decades. Ford believes that motivation is not only a pattern for pursuing goals, but also, it is the force that energizes, directs, and sustains behaviour among students [35]. In line with this idea, Fredricks et al consider learning motivation as the main power for educational success [36]. Thus, it plays a vital role in learning processes. Motivational theories that can explain students' motivation for learning vary. One of theories that suits the purpose of this study is that of intrinsic vs. extrinsic motivation. This theory explains that the source of motivation are compulsion and punishment [37]. There is also a relationship between intrinsic and extrinsic motivation. Tohidi note that a student may be extrinsically motivated at the initial stages of the learning process [38]; however, they may become intrinsically motivated as they go deeper in the learning process. On the other hand, Newman pinpoints motivation as an integral aspect of PBL which stimulates contextualized learning [39]. The concepts evaluated in this study through the questionnaire refer to achieving good grades, enjoying learning, time management, confidence in learning, academic success, competing others, and achieving a suitable job in future. These concepts refer to both intrinsic motivation (enjoyment, confidence in learning), and extrinsic motivation (achieving good grades, and acquiring a job).

IV. SOLUTION
The researcher argues that there is a logical relationship between the theories and the objectives of this study. Higher order thinking, collaborative learning, and experiential learning are among the main aspects of problem-based learning theory [40]. Motivation, on the other hand, is a learning feature investigated in numerous problem-based studies which has in many cases showed promising results [41]. The new aspect which is introduced in this framework in the interplay of problem-based learning, motivation and learning achievements in computer graphics is investigated. Information visualization theory explains that visualization is a cognitive skill. Thus, the researcher hypothesized that learning geometric concepts (i.e., transformation, scaling, and rotating) can be affected by a cognitive learning approach, i.e., PBL. This innovation is the contribution of this research to Problem-based learning theory and serious games.
The sampling method for selecting the participants in this study is non-random; believing that all basic and advanced computer graphic learners in Malaysia did not have the chance to be part of the study; therefore, this study is a quasiexperimental one.

B. CONTEXT
This study was conducted in the context of Asia Pacific University of Technology & Innovation known as APU in Malaysia during the second semester of the year 2019/2020.

C. PARTICIPANTS
The participants of the study were both male and female undergraduate students taking a computer graphics course at APU in semester 1, 2019/2020. These participants came from 3 different ethnicities of Malay, Chinese, and Indian. Their age range was between 18 and 22, and they had all gained their high school diploma degree. They all had at least 1 year experience in studying at APU in computer related courses and were all taking computer graphic course as an obligation to receive bachelor degree in computer games, multimedia technology, computer science and information technology. These participants formed to 2 groups, the experimental group (n = 24) and the control group (n = 26). The control group received traditional lecture while the experimental group received the treatment.

D. INSTRUMENTATION
One of the challenges of this study was selecting an appropriate instrument to measure learning motivation. The main reason for such a difficulty was that the motivational traits in different subject areas are different and the items in some questionnaires may not be appropriate for all subject areas. Another limitation is that many questionnaires to measure motivation, as noted by Velayutham, Aldridge and Fraser are developed by psychologist; thus, the intricacies of a specific subject area may not have been considered in them [42]. To find the appropriate questionnaire of the purpose of this study, the research reviewed previously designed and used questionnaires. Unfortunately, a learning motivation questionnaire on computer graphics was not found and VOLUME 10, 2022 the researcher came to the conclusion that a questionnaire should be adapted from studies in similar areas. A number of questionnaires in different areas were found and investigated. Although each of these questionnaires have a merit, the researcher finally decided to use the learning motivation questionnaire designed by McCord and Matusovich [43] which were originally designed for the field of thermodynamic but its constructs were most appropriate to this study. The questionnaire consists of 26 items on extrinsic motivation intrinsic motivation, interest, attainment, cost, identification with academics, self-efficacy, and instrumentality. The questionnaire designed by McCord and Matusovich [43] was used in this study as the constructs in the questionnaire were related to problem-based learning. The designers consider their questionnaire the intersection between motivation and cognitive perspectives of learning. Another reason for selecting this questionnaire is that the questionnaire is very comprehensive. As noted by McCord and Matusovich, primarily, the researchers considered a wide range of motivational constructs and surveyed their participants and finally shortened their constructs and came up with 8 constructs [43]. The third reason for selecting this questionnaire is the nature of the constructs under investigation. The constructs include: extrinsic motivation, intrinsic motivation, interest, attainment, cost, identification with academics, self-efficacy, and instrumentality. These constructs are relevant to the concept of learning, whereas the constructs found in the other questionnaires seemed to be more general, and not as related to the concept of learning as the construct in this questionnaire.

E. PROCEDURE
The study was conducted in the first semester of the educational year 2019-2020 and lasted for 8 weeks. In order to conduct ethical research and to comply with rules of research, a consent form was prepared and given to the all participants prior to the main study.
The first phase included piloting the Likert scale questionnaire on learning motivation by McCord and Matusovich [43]. The aim of this pilot study was not to reduce the subscales and the items of the questionnaire, as the items were already designed and validated by the designers. However, the questionnaire was administered to 30 undergraduate students studying at APU in order to check the reliability and validity of the questionnaire prior to the main study, and to find out whether or not there is a need to attempt to enhance the students' motivation. To make sure the questionnaire suits the purpose of the study, it was emailed to 5 experts in the field of problem-based learning. The experts were asked to rate each item of the questionnaire. Using the results Cohen Kappa was measured for each item of the questionnaire. As the score gained for each item was 0.8 to 1, it was observed that the questionnaire has content and face validity. In order to check the construct validity of the questionnaire and as the questionnaire was adapted from an original study, exploratory factor analysis (EFA) was run. EFA measures construct validity by underlying relationships between measured variables and is based on a common factor model [44].
As noted by Gliem and Rosemary, when a Likert scale questionnaire is used, it is imperative to use Cronbach Alpha to measure internal consistency [45]. Also, as noted by Adeniran, in case the Likert Scale questionnaire is used to measure multiple concepts, Cronbach Alpha should be calculated for each concept (subscale) [46]. Therefore, to measure the reliability of the questionnaire, Cronbach alpha was measured for each subscale. Table 2 shows the results of reliability test. As can be observed in Table 2, the reliability index for all 8 subscales of the questionnaire is above, .74. Based on George and Mallery, this indicates that the questionnaire is a reliable one [47].

F. TREATMENT
The main and most significant phase of the study is the treatment phase. The treatment last for 8 weeks and a session per week. Each session of the treatment last for 1 hour and 45 minutes. While the participants in the control group experienced the conventional education for computer graphics as APU syllabus, the participants in the experimental group went through the problem-based SG designed for the course. The number of session and the amount of time spent for education in both groups was equal.
To present the computer graphics concepts to the students and by considering the aforementioned features for a PBL design for SG, the Immersivio game was designed based on the Immersivio framework. The overall goal of each level of the game is to solve a puzzle by using provided tools and programming which is a problem presentation based on Hmelo-Silver's PBL tutorship [48].
The focus of the game is on APU's syllabus. The game was sent to experts in both fields along with an evaluation form. The evaluation form measured each item on the scale of 1 (Completely Disagree) to 5 (Completely Agree). Using the data collected from the experts, Cohen Kappa index was calculated. As in both cases, the Cohen Kappa index fell between .8 and 1.00, it can be assumed that the game correctly measures knowledge of computer graphics test in a problem-based game format.
The game is designed in 3 levels and each level these which all aforementioned geometric concepts. The procedure of the game is shown in the figure 1. The player should initially interact with the game. Following the game user interface, the player interacts with 3 different levels. As an example, the demographics of level 3 is demonstrated in figure 2. As can be seen in Figure 2, interactive and fixed prefabs are used. This figure visualizes the game at Level 3 in a solved state which leads the student to goal achieved page and shows the final results. At the same time the participants game data will be collected and saved.

VI. DATA ANALYSIS AND RESULTS
The data collected in this study were analysed through multivariate ANOVA (MANOVA) which assumes normality of the data, homogeneity of variances of the groups and homogeneity of covariance matrices. The normality of the data was probed using skewness and kurtosis indices and their ratios over the standard errors. The absolute values of the ratios were lower than 1.96, hence normality of the data was assumed.

A. PRE-TESTS COMPARISION
A multivariate ANOVA (MANOVA) was run to compare the two groups' means on the pre-tests of components of motivation; in order to prove that they were homogenous in terms of their level of motivation prior to the administration of the treatments. MANOVA has two specific assumptions; i.e., homogeneity of covariance matrices and homogeneity of variances. The assumption of homogeneity of covariance matrices, which is tested through the Box's statistics, requires that the correlations between any two components be roughly the same across the two groups. The non-significant results (Box' M = 27.52, p = .962) indicated that the assumption of homogeneity of covariance matrices was retained.
MANOVA also requires that the groups' variances be roughly the same; i.e., homogeneity of variances. Based on the results displayed in Table 3 Table 4 displays the main results of the MANOVA. Based on these results (F (8, 40) = .322, p = .953, Partial η2 = .061 representing a moderate effect size) it can be concluded that there were not any significant differences between the two groups' overall means on the pre-tests of components of motivation. Table 5 displays the descriptive statistics for the experimental and control groups on the pre-tests of components of motivation. Based on these results and the MANOVA results for comparing groups on each of the components of motivation in the table, it can be concluded that: A: There was not any significant difference between the experimental (M = 3.68) and control (M = 3.80) groups' means on pertest of extrinsic motivation (F (1, 47) = .395, p = .533, partial η2 = .008 representing a weak effect size).    G: There was not any significant difference between the experimental (M = 4.22) and control (M = 4.18) groups' means on pre-test of self-efficacy (F (1, 47) = .106, p = .746, partial η2 = .002 representing a weak effect size).

B. POST-TESTS COMPARISION
A multivariate ANOVA (MANOVA) was run to compare the two groups' means on the post-tests of components of motivation; i.e., extrinsic motivation, intrinsic motivation, interest, attainment, cost, identification with academics, self-efficacy, and instrumentality in order to probe the nullhypothesis raised in this study. Besides assumption of normality, MANOVA also assumes homogeneity of covariance matrices and homogeneity of variances. The assumption of homogeneity of covariance matrices, which is tested through the Box's statistics, requires that the correlations between any two components be roughly the same across the two groups. The non-significant results (Box' M = 32.93, p = .864) indicated that the assumption of homogeneity of covariance matrices was retained.
MANOVA also requires that the groups' variances be roughly the same; i.e., homogeneity of variances. It can be claimed that the assumption of homogeneity of variances was met on post-tests of extrinsic motivation (F (1, 47 There is no need to worry about the violation of this assumption. In case this assumption is violated, Tabachnick and Fidell suggested reducing alpha level [49]. They noted, Violations of homogeneity usually can be corrected by transformation of the DV scores. Interpretation, however, is then limited to the transformed scores. Another option is to use untransformed variables with a more stringent a level (for nominal a, use .025 with moderate violation and .01 with severe violation). The results of the MANOVA; i.e., Table 7, to be on the safe side, were reported at .01 levels of significance. Table 8 displays the main results of the MANOVA. Based on these results (F (8, 40) = 30.48, p = .000 < .01, Partial η2 = .859 representing a large effect size) it can be concluded that there were significant differences between the two groups' overall means on the post-tests of components of motivation. Thus, the null-hypothesis as ''problem-based SG did not have any significant effect on learning motivation by undergraduate students'' was rejected. Table 8

VII. DISCUSSION
In line with the findings of the current study, there are numerous studies that show motivation is a significant aspect of SGs [50], [51]. These studies have studied the effects of serious games on motivation in different disciplines; however, the consistency with regard to their results with the findings of the current study can be well observed. For example, Järvinen states that one of the main reasons for motivation in serious games is the self-functioning character given to the players [50]. He defines self-function as the degree to which the player understands the goals of the game and attempts to fulfil them. This situation results in engagement and eventually increases motivation. In the current study, engagement can be the result of increased cognitive load of the learning tasks. As stated by Jonassen, problem-based designs require the learners to analyse the learning problem on their own [52]; as a result, their cognitive engagement with the learning content increases. Cognitive engagement also results in self-efficacy [53]. This can explain why an increase in self-efficacy, as one of the main components of motivation, was also observed in this study.
Elsewhere, Watson and Lipford reported that SG is an effective tool to increase the students' motivation to learn computer science [54]. They made this conclusion by studied students at a software computer engineering course who played an SG to learn their course materials and were even more eager to continue optional levels of the game. Although the findings of their study in congruent with the results of the current study, they did not look into the components of motivation as was done in this study. Thus, the current study had a more detailed look at the concept of motivation.
Westera who focused on serious games and motivation asserts that the SGs that use experiential-learning as the main learning procedure are more motivating than those which use non experiential approaches [55]. Problem-based learning is an experiential learning approach with its roots in Dewey's experiential learning procedure [56]. In this approach, the learners are not provided by lecturers on how to play; rather they should explore the learning problem and find the solution on their own. This explanation can justify why problembased SG affected the learners' motivation through using Immersivio framework.
Fukuzawa, Boyd, and Cahn explain the impact of problembased activities on the learners' motivation as a result of 1) self-directed learning, 2) collaborative learning, and 3) use of problem-solving skills [57]. They explain that such features of problem-based activities increase the learners' engagement with the learning content, and as a result, their motivation is increased. Similarly, Savin-Baden asserts that using critical thinking skills in problem-based courses increased the learners' engagement with the content and eventually affects their motivation [58]. The findings of the current study are in line with Fukuzawa, Boyd, and Cahn and Savin-Baden [57], [58]. In the current study, collaborative learning through problem-based groups was encouraged and the learners' problem-solving and critical thinking skills were put to work by asking the learners to analyse the learning procedure on their own. This process was facilitated by designing a game environment.

VIII. CONCLUSION
The motivation of the participants was sought using McCord and Matusovich questionnaire which has 8 distinct constructs [43]. The researcher observed significant improvements in all constructs including extrinsic motivation, intrinsic motivation, interest, attainment, cost, identification with academics, self-efficacy, and instrumentality. Using MANOVA analysis, it was observed that all constructs were significantly affected as a result of implementing the proposed problem-based SG in this study. Thus, it is concluded that problem-based SG can significantly affect students' motivation in learning computer graphics.
This study can have pedagogical implications for the lecturers who are involved in teaching basic and advanced computer graphic courses. As it was observed that the Immersivio framework-based games are also useful for teaching the students who are demotivated and are looking for a new approach to learning. University lecturers who find explaining abstract notions to their students difficult can also use Immersivio. This problem-based SG is based on psychological concepts such as higher-order-thinking and aims at giving the learners an experience of the end results. This characteristic reduces the complexity of abstraction notions in the field of computer graphics.
This study has a number of theoretical implications as well. Serious games, problem-based learning, and computer graphics are distinct areas of science. Records reporting the emergence of these three areas in one theoretical framework were missing in the literature. However, this study bridged the gap between these three areas and reported a problem-based SG for learning computer graphics. In this way, this study contributes to the theory of learning computer graphics through problem-based SG.
Several suggestions are made by the researcher to continue this line of research. As this study only focused on learning of geometric concepts such as transformation, other researchers can focus on the effects of problem-based SG on concepts such as object modelling and animations. Such research can contribute to the literature in the field.
More research is required to realize the dynamics of interactions between the players and the games. As of now, we have only looked at products, i.e., the effects of problembased SG on learning 3D computer graphics and motivation. However, processes such as the human-computer interactions should be subject to more research. This query can be fulfilled through qualitative studies.
By playing Immersivio game, the players will receive an automatic play-score. This score can be used as an index of their computer-graphics skill and can be used to assess knowledge of computer graphics among all students in the field and also can be used as raw data in future related researches.