Gamification and Computer Science Students’ Activity

Students’ participation and motivation are of great importance in their learning process. The decrease in attendance and difficulties in stimulating students’ activity makes it necessary to find new methodologies that can solve these problems. The use of game mechanics in non-ludic environments (Gamification) has begun to be of great interest in research, since it could increase the motivation and therefore the activity of the students. This study tries to verify if there is an existing relationship between gamification and a possible increase in student activity or between the cessation of gamification and student activity. In addition, it evaluates whether a greater student activity corresponds to a greater learning improvement. In the obtained results, no significant differences were found between the methodology or the activity with a learning improvement. According to the study, the simple use of gamified elements does not necessarily imply a solution to the problems posed. The students’ activity in a course of these characteristics does not imply a better learning improvement, the importance lies in the quality of the activity generated and not in the quantity.


I. INTRODUCTION
Teachers report a decrease in class attendance [1], along with difficulties in stimulating students' interaction and discussion [2]. It is reported that the most pernicious problems, such as plagiarism and deception, are increasing [3]. Student participation is an essential component of learning, which has a great impact on academic performance [4].
According to [5], student learning increases when they are more engaged and interested in the learning process. By providing comments on student performance and current actions, a significant increase in student participation is achieved in any learning activity [6]. In [7] they found that adaptive feedback based on the collaborative behavior of students improved both student performance and their level of commitment.
Reference [8] argues that technology offers new ways of learning by providing authentic learning environments that The associate editor coordinating the review of this manuscript and approving it for publication was Chin-Feng Lai .
improve student learning experiences. Student participation improves when learning through technology [9]- [11]. Using technology is one of the potential ways to diversify the learning experiences of young people in the classroom and promote active learning [12], [13].
In [14] it was possible to improve the commitment and interaction in the classroom and provide a more individualized learning experience for the students using an application. Reference [14] supports the notion that digital educational games are useful tools to break down barriers between students and change classroom dynamics by helping young people to be active and collaborative participants in their learning.
Nowadays, the use of Learning Management Systems is widely extended in order to administer, distribute and control training activities from a web server. Moodle has been chosen for the implementation of the course used in this study, as in previous experiences [15]. It includes many indicators that measure the activity of a student, such as the number of publications, reading and participation in forums, VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ the access to tasks or the number of logins [16], [17]. The advantages of using Moodle include sharing information and providing material to students, announcing and collecting course assignments, conducting tests and publishing online [18], [19]. Moodle plays a crucial role in providing flexibility to support students' requirements, commitment and motivations [20], [21].

A. GAMIFICATION
Throughout history, games have been a way to entertain and have fun. The games have different objectives, some of them attainable in the short term in such a way as to give the players a sense of progress. Advances in technology allow us to enrich the games through different elements such as instant feedback or collaborative participation without the need for participants to be in the same place. Video games are part of a multidisciplinary, growing and leading industry that attracts talented programmers, designers and artists [22]. Modern students are growing in an era of interactive media and video games, so classroom gamification can be attractive and motivating [23]. The term gamification was used in 2002 by Pellin [24], it is commonly defined as the use of game design elements in non-ludic contexts [25]. Gamification tends to improve student engagement in virtual learning environments by using various elements of the game, such as badges, points, levels and leaderboards [26]- [28].
The use of gamification has become a highly debated topic [29]- [31]. The mere use of game elements in activities does not guarantee interactivity and commitment, since it will depend on its strategic use in relation to the problem, the educational content and the target population [30]. Developing a complete video game is very complicated, but applying gamification is much simpler, which increases the interest of academic research [30].
Companies and trademarks have successfully used gamification to encourage user activity, increase social interaction or support user engagement [32]. According to [33] people are supposed to be more engaged and more productive when they play games. Some studies [34]- [36] consider gamification as a way to transform education because of the potential it has to increase motivation and commitment.

B. GAMIFICATION IN EDUCATION
The literature review showed that digital games are motivating and educationally effective [37] and improve the effectiveness of science in particular [38]. Some studies claim that digital educational games have improved learning and knowledge acquisition [32], [39], [40] in fact, most experimental studies show that gamification has a significant impact on motivation [41], engagement [26], [42], [43] and learning outcomes [44].
However, the results of the use of digital games for learning should be treated with caution [51]. There have been concerns about the lack of rigor and empirical studies that have investigated the results of learning and digital educational games [52], [53], in addition, some research has reported negative effects on the results, such as reduced empowerment, engagement and motivation [54]. In [47] it was revealed that the Digital Game-Based Learning activity had no significant effects on the learning motivation and effectiveness of the students for the learning of science.
A lot of research and work such as [61] and [62] on student participation in the classroom has been carried out, but there are still difficulties in keeping students involved in their activities. Reference [63] states that it is complex for students to develop levels of participation to achieve their maximum learning potential. Understanding whether gamification can be used to involve students and improve their academic learning outcomes is a relevant and practical issue [64].
The results of the review [65] show a promising potential for Digital Game-Based Learning, particularly in the area of content understanding. However, the results of [65] also suggest that there is a need to provide additional research to obtain a more complete picture of the educational effectiveness of Digital Game-Based Learning.
Gamification in education is ''a serious approach to accelerate the learning experience curve, teaching complex subjects and systemic thinking'' [66], but there is little solid empirical evidence, if any, of the effectiveness of Gamification in education.

C. STUDY OVERVIEW
Reference [67] mentioned the topic that gamification research is maturing, transitioning from fundamental ''what?'' and ''why?'' questions to more differentiated questions about the implementation of gamification: ''how?'', ''when?'', and ''how and when not?'' Maybe this should be the direction of future studies.
In recent years, there are studies focused on education and gamification such as [68] that determine whether students would perceive the gamification activities in a positive light, [42] that maps game elements in a well-known generally accepted Learning Management System, or [69] focused on studying the impact of each element in order to investigate the behavioral outcome of game elements in educational environments. We also found research related to student participation such as [70]; that at combining gamification techniques and learning analytics to improve the engagement in University courses or [71] that test the effectiveness of gamifying activity breaks (AB) to enhance student participation, enjoyment, and confidence during AB in low-income schools.
If we expand the search to previous years, we find the review of [72] that mentions [73] which examines activity patterns after the elimination of a point-based incentive. Reference [73] found that the removal of points and badges reduced overall participation in a social software system implemented within a company. There are very few references to the moment when gamification stops, so we detect a lack of knowledge in this topic.
This study focuses on bringing to light the aspects that affect student participation and the possibility that the use of gamification can involve students. It seeks to verify whether the following assumptions are true: • H1: There is a relationship between the use of gamification and the students' activity.
• H2: A great student activity reflects a greater improvement or higher grades.
• H3: After taking a test, the grades obtained affect future activity in the course.
• H4: The cessation of gamification affects student activity.

II. METHODOLOGY A. METHOD
The method developed for this experience is based on a course designed as part of the database subject taught in the first semester of the first computer engineering course. The experience was carried out during a month in the middle of the first semester. The Moodle platform was used for its implementation. The course was about the E \ R model and the relational model.

B. SAMPLE
The average age of students in the first year of computer engineering was around 20 years. In the designed course, 190 students were registered and distributed in two groups (IBD1 and IBDA) randomly. The IBD1 group started with 96 students, but only 85 performed tasks. The second group, IBDA started with 94 although those who participated were 84. Throughout the course, three tests were carried out. The first tests were made before starting, as a level test (conducted by 85 IBD1 and 84 IBDA students), the second test took place at two weeks (performed by 77 IBD1 students and 77 students IBDA) and the third test was done at the end of the four weeks (done by 69 IBD1 students and 70 IBDA students). Throughout the course there were 77 dropouts of the 190 registered students.

C. STUDY DESIGN
The study used an experimental design between subjects in which the students were randomly distributed in two groups (IBD1 and IBDA). The characteristics of the students such as age, sex or knowledge of the subject were not used for distribution. The duration of the experiment was four weeks. The IBD1 group began as an experimental group having gamified elements in its course and the IBDA group began as a control group. At two weeks, in the middle of the experience, IBD1 became the control group, ceasing to be gamified and IBDA became gamified and therefore the experimental group.
In the first test, the previous knowledge of the students was analyzed. In the second one, which took place at two weeks, the evolution of IBD1 (gamified) and IBDA (control group) was analyzed. The final test checks the evolution of IBD1 after two weeks of being a control group after having been gamified from the beginning. On the contrary, in the last test IBDA students went from being the control group during the first two weeks to being the experimental group for the last two.
The experimental group, unlike the control group, enjoyed a gamified course with the following typical elements of gamification: • Badges: Distinction that has been awarded to overcome a milestone throughout the course.
• Feedback: Immediate response provided when performing tasks.
• Missions: The tasks have been grouped into missions.
• Points: Certain points are awarded when performing a certain action or delivering a specific task.
• Levels: Each level requires a certain number of points -As points are obtained, users will level up.
• Leaderboard: Table showing the list of all participants. You can see the participants, their score and their level.
• Time limit: Time set to complete a task.
• Blocked content: In order to perform some course tasks, it was required to complete some previous task.
To be able to use all these elements, the plug-in called GameMo was used, which allowed the integration of everything under the Moodle platform [74].

D. LOG FILES
Once a user registers on the Moodle platform, they begin to collect their activity records. These data are not handled for evaluation purposes but have been used to obtain an activity Feedback of the course. Log files store information about the login, access to resources, attempt at a test, participation in questionnaires, etc. The platform collects every user interaction on the web. Each click generates an entry that specifies user and action taken. Thanks to these logs, there is extensive information about student participation.
As the log files provide information about a student's last login, it is possible to know when they logged in and the period since the last login. The data on access to the tasks or the forums is also stored, although it cannot reveal whether the student has really read the content in depth or, on the contrary, has simply opened it without paying attention to it. The system only registers if it was clicked, but it cannot track the scrolling behavior; therefore, there is no guarantee that students would read the content after accessing. The data on the attempts and results of the students' tests were also recorded, which makes it easier to observe more closely the performance of the students and draw conclusions about improvement. VOLUME 8, 2020 After completing the course, students' records were downloaded and stored for analysis. The system allowed the identification of the user activity records of the platform with the surveys and tests carried out, allowing a deep analysis of the evolution of each student.

E. EVALUATION TESTS
During the experience three tests were performed to measure the student's evolution. The content and the difficulty to be evaluated in the tests increased as the work progressed.
The first test consisted of 10 test questions to solve in 20 minutes. These questions were generally related to the subject. The first 7 questions were theoretical and the last 3 consisted of selecting the Entity Relationship model corresponding to a statement.
The second test consisted of 10 test questions to be solved in 30 min. The time was increased compared to the previous test, due to the increase in difficulty and effort required. It consisted of four theoretical questions and six questions about selecting the entity relationship model corresponding to a statement (more complex than those of the first test).
The third test had to be done in 40 min. Like the previous ones, it was a multiple-choice test. It had two theoretical questions, four questions about selecting the entity relationship model corresponding to a statement (like the second test) and four questions about converting an entity relationship scheme to a relational scheme.

III. RESULTS AND DISCUSION
In this study, the activity records of the students stored in the log files have been thoroughly extracted and analyzed. The scores of the tests performed by the students have also been collected.  Fig. 1 shows information of 4 groups:

A. STUDENT ACTIVITY PER HOUR
• IBDA 1: Behavior of the IBDA group in the first two weeks when it was a control group.
• IBDA 2: Behavior of the IBDA group in the last two weeks when it was gamified.
• IBD1 1: Behavior of the IBD1 group in the first two weeks when it was gamified • IBD1 2: Behavior of the IBD1 group in the last two weeks when it was a control group. ''IBDA 1'' and ''IBD1 1'' have higher participation values as there are more students, since in the last two weeks of the course in which the data of ''IBDA 2'' and ''IBD1 2'' had already been taken, they counted student dropouts.
If we compare the first two weeks or the last two, both in the IBDA group and in the IBD1, we find very similar trends. The hours with more student activity were 12 in the morning and in the afternoon between 16 hours and 22 hours. This behavior is similarly reflected in both IBDA and IBD1, regardless of the methodology applied.
From 22 hours, the activity drops to practically zero until 3 in the morning. At 9 in the morning it increases again. This time slot corresponds to the hours in which the students are sleeping so, it is logical that in these schedules there are few accesses.
From 9 in the morning it begins to increase until it reaches the peak at 12, possibly, students begin to get up and do their homework. From 12 o'clock there is a descent to 15, surely because 12 o'clock corresponds to the mealtime. From 15 hours the activity increases, this is maintained during the afternoon until 22 hours. This time slot corresponds to the afternoon hours that students dedicate to homework.

B. STUDENT ACTIVITY PER DAY
Another interesting fact to note about the activity is the days of the week. Fig. 2 shows the behavior of the 4 groups mentioned above. We can observe that in the first weeks there were more marked differences in the activity prevailing on Wednesday. However, in the last weeks, there is no day of the week that stands out from the rest. The most striking thing is that the behavior of both groups, regardless of the methodology, was similar, and the activity could be affected by events outside the course. It is possible that the workload of other subjects or different commitments on the part of the students makes for more activity on some days than on others.

C. STUDENT ACTIVITY PER WEEK
Finally, to compare the two groups in the graph, we observed in fig. 3 the activity generated by IBDA and IBD1 during the 4 weeks. The behavior is very similar in both groups. In the  first two weeks, the activity was greater, but from the third week there are dropouts and a decrease in activity. The fourth week is the one that registers with less difference and less activity. Regardless of the methodology applied, the graph behaves the same way, therefore, it is possible to say that gamification, in this case, has not produced a significant increase or decrease in activity. There is only a significant decline over time.

D. STUDENT ACTIVITY AND TESTS
In the tables of this section, the IBDA and IBD1 groups have been divided into three: Students who scored below average (BELOW), students who scored around the average (MED) and students who scored higher than the average (HIGH). The average participation and standard deviation of the three groups are shown on the tables. The last column shows the results of the t-test between the BELOW group and the HIGH group, so it reveals how the differences in participation between both groups are significant.

1) STUDENT ACTIVITY AFTER TEST
In the first row of table 1 the participation of IBDA students is seen in the two weeks following the first test, grouped by the results obtained in this. In the second row they are grouped by the results obtained in the second test and the activity of the following two weeks. The last rows show the IBD1 data in the same way: Although the HIGH group had slightly more activity per user than the BELOW, the regular group had worked the most.
The data in the table shows that a good or bad result in the test does not imply that the student will participate in the following weeks. Observing the results of the t-test, we found no significant differences in IBDA or IBD1 (considering 0.05 as the top value).
In addition, if we consider that IBD1 was only gamified for the first two weeks, and that IBDA was only gamified after the 2nd test, we find that participation in the gamified groups does not have significant differences from the control groups.

2) STUDENT ACTIVITY BEFORE TEST
Although, according to the data obtained, a test result does not affect future participation, we have the question of whether previous participation may be related to good results. Table 2 shows the average participation, standard deviation and t-test between the BELOW group and the HIGH group of both IBDA and IBD1. In the 2nd test row the students are grouped by the results of the second test and the participation is from the two weeks prior to the test. In the 3rd test row, the averages of the two weeks prior to the third test are compiled by grouping the students according to their results in this. Except in the 2nd test of IBD1, the data in the table reflects a greater activity in the HIGH group than in the BELOW group, although in the case of IBD1 the highest activity is recorded in the MED group. Therefore, we cannot establish a relationship between previous participation and the result obtained in the test. Only a significant difference is obtained between the BELOW group and the HIGH in IBDA in the third test, and the participation in the previous two weeks in which they were gamified. In IBD1, no significant differences can be seen, either when it was gamified (2nd test), or when it was a control group (3rd test). We can say that no clear references have been found in this study between participation and the performance of subsequent tests, regardless of methodology. Table 3 compares the activity of the intermediate weeks from the first to the second test and from the second to the third. Both IBDA and IBD1 are compared. They have been subdivided into three groups based on the improvement between the two tests: those that have evolved below the average (BELOW), those that have evolved around the average (MED) and those that have improved above the average (HIGH).

3) IMPROVEMENT AND STUDENT ACTIVITY
According to the t-test and taking 0.05 as the maximum value to identify a significant difference, we did not find significant differences in the data analyzed in the table. It was VOLUME 8, 2020 observed that the students who improved less during the first two weeks are the ones with the highest activity, whereas the ones with the least improvement in the last two weeks are the ones who registered the least activity, therefore, there is no relationship between participation and the improvement obtained.

4) STUDENTS ACTIVITY WHEN GAMIFICATION STOPS
IBD1 was gamified in the first period, while in the second it ceased to be. As we can see in figure 3 in the first and second week, the activity while the students were gamified was more extensive than in the third and fourth week in which the activity descends. If we pay attention only to IBD1 data, we would reach the same conclusion as [73], however, this behavior is very similar in the IBDA group that begins without gamification and ends gamified. Therefore, in our case, the reduction in activity cannot be related to the cessation of gamification. In [73] they do not have a control group to contrast the results, it is possible that their results may be due to other causes such as the loss of the novel effect and not the cessation of gamification. No direct relationship has been found between the cessation of gamification and activity.

IV. LIMITATIONS
This study focuses on the analysis of the effect that the use and the cessation of gamification can have on the student's activity. Therefore, only information about the generated activity in the course is included in the results presented. Although motivation and collaboration among students are interesting, they are no studies focused on that area. Nor is it intended to evaluate the Moodle platform or the course design. We want to compare whether there is a relationship between the use of gamification and the activity.
A Moodle course has been used parallel to the students' classes, there are a test group and a control group with the same tasks. The data collected are from students in the second year of computer engineering. The results may vary if the sample changes or the course was design in a different way. It is possible for future experiments with different samples of subjects, or a different course design to contrast the results obtained here.
The E\R model and the relational model were specifically chosen for the possibility of creating exercises that would help to learn the theoretical content of the subject of Databases. The study could be replicated in another subject. In fact, it would be a good idea to try to test whether the results obtained are the same in another sample of students or in different courses.

V. CONCLUSIONS AND FUTURE WORK
In this study, no significant difference in activity was observed between gamified or non-gamified groups. No direct relationship was found between cessation of gamification and students' activity (H4). Regardless of the methodology, student participation was similar in the two groups studied throughout the 4 weeks of the experience (H1). A drop in student records was detected due to dropouts that occurred as well as a decrease in participation over time.
In longer experiences, this aspect should be considered.
The activity of the students on the platform, two weeks before or two weeks after a test, could not be related to the grades obtained (H3). This reflects that, depending on the student, greater or lesser activity may be necessary to obtain good grades, but greater activity does not guarantee a significant difference in learning. In addition, the grades obtained by the students between one test and another have not reflected a clear relationship that greater activity implies greater improvement. The amount of time that the students spent on the platform does not affect learning as much as quality (H2).
Therefore, this study has not found a clear relationship between the activity and the methodology used, nor between the activity and learning. It is possible that there are other factors that affect the improvement of learning, surely in a longer experience it would be possible to see more marked differences, but in experiences of short duration such as this study, these have not been significant. As future work, it is necessary to analyze a longer experience to see if the reduction in student activity increases over time, and to checking if significant differences in participation can be found by applying gamification for a longer time.
MIGUEL GARCÍA-IRUELA worked as a computer science engineer for four years and as an instructor for five years. He is an Adjunct Professor with the Department of Telematics and Computational Systems Engineering, Universidad Rey Juan Carlos, and a Volunteer Training Teacher with IES Clara Del Rey. His main research areas include gamification, educative technologies, volunteer training, and personalized learning. His main research areas include human-computer interaction, brain-computer interfaces (BCI), emotions recognition, health systems, multimedia information retrieval, and sketch recognition. He is a Senior Member of the ACM.
RAQUEL HIJÓN-NEIRA received the European Ph.D. degree in computer science, in 2010. She worked as a computer science engineer for five years and as an instructor at the university for 20 years. She has been a member of the Laboratory of Information Technologies in Education (LITE) since it's starting. She is an Assistant Professor with the Department of Computer Science, Universidad Rey Juan Carlos, Madrid, Spain. Her research areas are software for and innovation in programming education, educative technologies, gamification, and serious games.
TERESA CHAMBEL (Member, IEEE) received the degree in computer science from the Faculty of Sciences, University of Lisbon, Portugal (FCUL), with a final research project in AI expert systems for intelligent buildings with the LNEC, the master's degree in electrical and computer engineering from IST, with a thesis on distributed hypermedia, and the Ph.D. degree in informatics from FCUL, on video, hypermedia, and learning technology. She is an Associate Professor with the Department of Informatics (DI), FCUL, and a Senior Researcher of the LASIGE Research Unit. In her research areas, she has been teaching and researching, authored around 150 articles and book chapters, and participated in the organization and program committees of workshops, conferences, and journals. She was a founding partner of ONTV, having participated in the launching of Interactive TV in Portugal, and has been collaborating with the industry in the contexts of research projects and students' supervision. Her research interests include multimedia and hypermedia, human-computer interaction, creativity, immersion, visualization, accessibility, cognition and emotions, and well-being and digital arts. She is currently leading the Research Project AWESOME: Awareness While Experiencing and Surfing on Movies Through Emotions, the Executive Vice-President of the ACM TVX/IMX Steering Committee, and a Guest Editor of the special issue on Multimedia Alternate Realities of the Multimedia Tools and Applications journal (Springer).