Games and Rewards: A Scientometric Study of Rewards in Educational and Serious Games

In this study we provide a new viewpoint on the body of literature regarding rewards in serious and educational games. The study includes a quantitative bibliometric analysis of literature in this context from 1969 to 2020. The dataset from the Scopus abstract and citation database was analyzed with the Bibliometrix R library. The data set was manually cleaned to contain only the relevant articles and conference papers. The data was then categorized to match the common themes. From the remaining documents, the amount of annual numbers of publications is presented and the most contributing countries are shown. The most frequent terms from the abstracts and keywords set by the authors are presented, and a co-occurrence network is drawn from the same data. The results of this study reveal that the most occurring topics in this dataset are gamification, physical activity, health, game design, and game-based learning. New directions for research are provided as the most commonly used media appear to be video games and mobile devices in addition to the literature being mostly focused on theory and not practical application.


I. INTRODUCTION
Video games have a strong presence in global culture and market [1], [2]. Interest in video games is growing [3], especially among youth and young adults [4]. Gameplay offers learners opportunities to gain new information [5] and shows promise as an effective teaching platform [6]. Studies exist that assess the effectiveness of game-based instructional programs [1], some of which have been found to be motivating and effective [4]. There is little literature about engagement's effect on learning outcomes [5]. More research must be done on actual learning content than the visual aspect of educational games [7].
Educational games are referred to as 'serious games'; [8] states that ''serious games are digital games, simulations, virtual environments, and mixed reality/media that provide opportunities to engage in activities through responsive narrative/story, gameplay, or encounters to inform and influence for well-being and/or experience to convey meaning''.
The associate editor coordinating the review of this manuscript and approving it for publication was James Harland.
Game-based learning is described as playing games with the purpose of learning [9]. Research suggests that most of the research done in this context relates more to theory than practical application [10]- [12]. According to [13], game design elements can be categorized into nine core elements: personal profile, non-fixed structure, challenge, feedback, short cycle time, theme, competition, cooperation, and chat-based social network. The work in [13] bases these design elements on the self-determination theory, which aims to maintain intrinsic motivation by satisfying an individual's needs for autonomy, competence, and relatedness [14].
Some studies express concern that some games are designed to be neither instructional nor educational [1] nor show an increase in motivation for science or science-related work [6]. Developing a better understanding of the various activities educational games offer is key to understanding how they can be implemented to achieve the desired results [3]. More specifically, the elements producing enjoyment and increasing student motivation must be studied to better understand how learners can be kept engaged in gameful learning activities [5]. More qualitative studies are needed to explore the nature of engagement [3]. The education field in general would benefit from more interesting and stimulating educational software [4]. As the literature on games in the educational context is fragmented [15], this study provides a new viewpoint on how rewards are used in serious and educational games.
There are plenty of different categorizations for reward types in video games. [16] classified rewards into separate categories and this idea was further processed into six reward types: access, facility, sustenance, glory, praise, and sensory feedback [17]. [18] differentiates among eight different reward types: score systems, developable avatars, item granting rewards, resources, achievement systems, feedback messages, plot animations, and pictures, as well as unlocking mechanics. Similar rewards can be used outside of educational video games and have been studied in gamification intervention studies [19].
Gamification has been explained as ''the application of lessons from the gaming domain to change behaviors in non game situations'' by [2] and [20], describing it as ''transforming activities, systems, and services towards affording similar experiences as games are considered to afford''. Gamification is said to heavily support reward usage via points, badges, leaderboards, and levels [21]. Most gamification research is focused in design theory [22]. Gamification is often used in contexts where users are found in need of motivation such as education and healthcare fields [23]- [28].
Bibliometric studies in games and gamification have been done before, such as a healthcare-focused analysis about gaming [29], point systems in games for health [30], and gamification in education focusing on researchers, institutions, and themes [31]. The datasets were obtained by forming a search string through experimentation and combination as presented in [30]. Bibliometric studies such as [29] and [31] include various data points from the dataset such as co-occurrence of keywords, yearly term frequency and yearly production of research. The availability of information and number of applications available to mobile phones has increased, and the applications have diversified in the past two decades [32]. However, no bibliometric study has taken a holistic view of rewards in games, gamification, and serious games publications. To understand and further guide the growing body of literature in reward research in this field, understanding where the research is being done, what kind of research is being done, and what keywords are being used in research is important.
This article offers a new viewpoint to the reward body of literature in serious and educational games: a quantitative, bibliometric analysis of reward literature in this context from 1969 to 2020. By analyzing Scopus-based dataset. The objective is to report a comprehensive view of the literature that includes the important research themes, the timeline of growth in the body of literature and the most contributing countries.

II. METHODOLOGY
The data for this bibliometric study was retrieved on October 7th, 2021, from Scopus. The analysis was done with the Bibliometrix R library [33]. Similar to [30], words that are relevant or similar to the subject of research were collected and converted to search terms which were applicable to the database used. The search terms used were as follows: ( TITLE (( videogame * OR "video-game * " OR gaming OR game * ) AND (student * OR learn * OR teach * OR serious OR educat * ) AND reward * ) OR ABS ( ( videogame * OR "video-game * " OR gaming OR game * ) AND ( student * OR learn * OR teach * OR serious OR educat * ) AND reward * ) OR AUTHKEY ( ( videogame * OR "video-game * " OR gaming OR game * ) AND ( student * OR learn * OR teach * OR serious OR educat * ) AND reward * ) ) AND ( LIMIT-TO ( SRCTYPE , "j" ) OR LIMIT-TO ( SRCTYPE , "p" ) ) AND ( EXCLUDE ( PUBYEAR , 2022 ) OR EXCLUDE ( PUBYEAR , 2021 ) ) AND ( LIMIT-TO ( LANGUAGE , "English" ) ) Using this search term, 1779 document results were retrieved from Scopus. The timespan for the data ranged from 1969 to 2020. The retrieved publications were manually filtered to match inclusion criteria and research questions using abstracts, keywords, and titles. Articles and conference papers unrelated to rewards in games in educational contexts, such as machine learning, were discarded. After leaving only the relevant articles and conference papers, 429 documents remained. Most of the removed documents were related to machine learning subjects. Similar keywords set by authors were aggregated and then categorized. As seen from Table 1, the final data included 196 articles and 233 conference papers. Author keywords that were similar (e.g., ''video games'', ''video game'', ''video gaming'', and ''video-games'') were combined. Some keywords such as ''vocabulary'' were created from similar keywords such as ''Arabic vocabulary'', ''vocabulary learning strategies'', and ''vocabulary acquisition'' to obtain larger data points for analysis in certain contexts. Another example would be the category ''user experience,'' including the author's keywords ''user experience'', ''user experience evaluation'', and ''user experience gamification''.
Unigrams, bigrams, and trigrams were extracted from the contents of the abstracts. Bigrams and trigrams containing irrelevant information such as publishing companies were removed from this data set. The Louvain clustering algorithm was used to further analyze the co-occurrence of author's keywords; The Louvain method of community detection provides a way to detect communities in large networks [34].

III. RESULTS
As seen in Figure 1,the period between 2005 and 2020 has seen significant growth compared to previous years regarding the number of articles per year. It can also be seen from   Germany, Italy, and Portugal. The USA had 1721 citations, whereas the citations from the remaining countries ranged from 158 to 422. As seen in Figure 2, the largest contributing countries for scientific production were the USA, the UK, and Canada. The study shows that the authors from these countries did not work alone, as Figure 3 shows a visualization of collaboration among the nations. The most frequent collaboration happened between the USA and Canada, as well as the UK and Greece.
As seen in Table 2, the most commonly found word pairs from the abstracts were ''game design'', ''gamebased learning'', ''game elements'', ''intrinsic motivation'', and ''video game''. Words referring to learning, such as ''game-based learning'' and ''learning process'', account for 22 percent of the occurrences in 2.. The words ''video game''  and ''video games'' account for 18 percent of the bigrams in the ten most frequently found bigrams from abstracts. Computer science is the only field on the list. The fourth most occurred bigram is ''intrinsic motivation''.

B. AUTHOR'S KEYWORDS ANALYSIS
The ten most frequently found words from the author's keywords are ''gamification'', ''learning'', ''motivation'', ''education'', ''serious game'', ''reward'', ''game-based learning'', ''mobile'', ''game'', and ''interaction''. Gamification has the most occurrences in the keywords set by authors, seen in Table 3. This keyword accounts for 22.4 percent of the occurrences in Table 3. This keyword accounts for 22.4 percent of the occurrences in Table 3. Words containing the word ''learning'' are ''learning'' and ''game-based learning'', making up 20 percent of the occurrences (see Table 3). Motivation has the third most occurrences on the table of keywords set by authors, consisting of 11.7 percent of the occurrences.
The ten most frequently found keywords set by authors (Table 3) did not see much popularity before 2004. The most occurring keyword, ''gamification,'' first appeared in number during 2010, after which it saw a significant rise in occurrences, seen in Figure 4. Other words following a similar trend but are not as popular are ''learning'', ''motivation'', ''education'', and ''serious game''. Game-based learning has seen a significant number of occurrences during 2016, 2019, and 2020, compared to previous years.
The Louvain clustering algorithm was used to find thirty nodes. The outcome was four different communities from the author's keywords. These communities can be seen in Table 4. A co-occurrence network (see Figure 5) was formed from these four clusters. The largest nodes in the red cluster are gamification, motivation, learning, rewards, and game-based learning. Gamification can be seen as strongly connected to learning, game-based learning, and motivation. The second-largest nodes can be found from the blue cluster: education, mobile, and game. Education and gaming share a strong connection within the blue cluster. The green cluster has several smaller nodes, but the largest are serious games, game design, and interaction. The lavender cluster shows connections among rewards, competition, and behavior, as well as educational games and language.

IV. DISCUSSION AND CONCLUSION
This article set out to provide a timeline of growth and the largest contributing countries, as well as popular and common themes in article abstracts and keywords set by authors in the body of literature about rewards in serious and educational games.
The number of articles produced per year has seen significant growth from the beginning of 2000. The peak years for production were from 2016 to 2020. The largest contributors to the scientific production were identified as the US and the UK. Collaboration between countries was mapped; the highest collaboration was found to be between the US and Canada.   The most co-occurring topic in this data set is gamification. According to [32], the rise in gamification studies is the popularity of mobile phones with internet access, social media, and the widespread application of applications to said devices. Table 3 supports this view as the word ''mobile'' appears on the most frequently found keywords set by authors. The importance of the computer game industry and interest in learning from employee and customer behavior has also been suggested as one reason for the popularity of gamification studies by [2]. The high co-occurrence of publications related to gamification, seen in Table 3, could also be because most gamification efforts heavily focus on points, badges, leaderboards, and levels, [21] which are types of rewards, and rewards are the key subject of this study.
It has been suggested that the actual learning outcome is not a key area in studies in the engagement context [5]. In the co-occurrence network of the author's keywords, the keyword gamification was found to be connected to other nodes in the same cluster: learning, motivation, game-based learning, and engagement. This suggests gamification may have been studied in increasing motivation and engagement more than in actual learning context.
Most gamification studies between 2016 and 2020 were done in the design context [22]. The most covered concepts were education, teaching and learning, engagement, motivation, behavior change, and gameful design [22]. The data seems to support this view as the most frequently found keywords set by authors (Table 3) has similar keywords such as education, learning, and motivation. The emphasis on design in the recent literature could be a response to research done in previous years, suggesting that these endeavors produced games that are neither instructional nor educational [1] and provide no increase in motivation [6].
According to [21], many gamification studies are conducted in computing, engineering, or social science subjects. The bigram computer science appears on the most frequent word bigrams found from the abstracts, which support this view. Some studies suggest that physical activity is the most commonly gamified context [23], [35], [36]. The claim gets further support in this study as physical activity is high on the list of occurrences in the abstracts.
Some studies argue that even though there is a lot of talk about educational games, few actual studies about the creation of educational video games [10]- [12] exist. The most frequent word bigrams found from abstracts supports this claim as some of the most often used word pairs from abstracts are game theory-related, such as (but not limited to) the game, design, game design, game elements, learning process, and game mechanics. It has been argued that understanding the various activities in the educational gaming context is key for achieving wanted learning outcomes [3]. The data seems to suggest that the body of literature is growing in this field as game design and game elements are among the most frequent word bigrams in Table 2. Although studies that use and study different rewards in gamification intervention studies exist [19], [31], bibliometric studies [29] and point systems for health [30] it is interesting to note that even though game design elements represent high co-occurrence in this data set, no specific reward types such as access, facility, sustenance, glory, praise, or sensory feedback (as categorized by [17]) is seen in this data set. Intrinsic motivation is near the top of Table 2, but the bigram extrinsic motivation has not reached a similar occurrence in the data set.
The study is limited to a single database, Scopus, which includes almost all Web of Science content in addition to more technical coverage of articles relevant to our research question. To ensure relevance of the included data, we have manually filtered every manuscript. While the manual filtering may introduce an element of subjectivity, it has significantly improved the signal (relevant articles) to noise (articles irrelevant to the research question) ratio and therefore, the benefits outweigh the possible subjectivity. A single database is arguably less inclusive, however, since each database has its own citation counts, combining databases brings imbalance among the included articles. The search terms used in this research can be used to replicate the study, which offers interested researchers the opportunity to verify, confirm or refute some or all of our findings.
This study indicates that the body of literature on rewards in serious and educational has seen a significant rise from pre-2005 to 2020. Gamification, game design elements, learning motivation, and applications for health and physical activity are the most relevant topics in this data set. The most commonly used platforms to reach these goals are video games and mobile devices. This study has contributed a general overview of the literature on rewards in serious and educational games. With this knowledge, the direction of the rigorous scientific research of this literature can be directed in areas that have not yet been saturated and where more knowledge can be acquired from new fields in the reward context of serious and educational games.
As the body of literature in the motivation context of serious games and gamification is diverse, for example [4], [7] suggest these are not so motivating. However, another study suggests they are quite motivational [1], and [6] reports no change in the motivation of learners. Notably, the word ''motivation'' is one of the most frequently occurring words in abstracts and keywords set by authors. As the literature on the subject is diverse, more research on the application of rewards, platforms other than mobile, and studies outside the computing, engineering, or social sciences fields, are necessary to draw meaningful conclusions from the existing body of knowledge.
Future research directions in this knowledge domain are topics which are not highly represented in this dataset. For example more research in the context or rewards could be added in creativity, low carbon education and sustainable development goals in addition to other topics that are underrepresented in the dataset. The timing of rewards (for example during or after a task) is also an interesting direction for this subject as there is little representation about this topic. Another direction could be researching the difference between rewarding based on progress in the task in addition of rewarding from successfully completing a task. As theory related subjects seem to be vastly more popular than practical implementations, we suggest that practical applications are presented in the body of research. Platforms other than mobile, such as virtual and extended reality would benefit from additional research as the formerly mentioned appears to be the most researched platform in the context of this study.

SONSOLES LÓPEZ-PERNAS (Graduate Student
Member, IEEE) received the bachelor's and master's degrees in telecommunications engineering and the Ph.D. degree in telematics engineering from the Universidad Politécnica de Madrid (UPM). Since 2015, she has been working as a Researcher with the Next Generation Internet Group (GING), UPM. She is currently an Assistant Professor with the Department of Computer Science, UPM. Her research interests include learning analytics, technology-enhanced learning, computer science education, and bibliometrics.
MOHAMMED SAQR received the Ph.D. degree in learning analytics from Stockholm University. He is currently a Senior Researcher at the University of Eastern Finland on artificial intelligence, big data in education, network science, and scientometrics. He is particularly interested in research methods, including network analysis, temporal networks, machine learning, process, and sequence mining, as well as temporal processes. He is also an active member of several scientific organizations and an academic editor in leading academic publications.
JUHO KAHILA received the M.Ed. degree from the University of Jyväskylä, Finland. He is currently pursuing the Ph.D. degree with the School of Applied Educational Science and Teacher Education, University of Eastern Finland. His research interests include children's learning related to digital games and metagame.

ROMAN BEDNARIK is currently an Associate
Professor at the University of Eastern Finland. His research interest includes human-computer interaction.
MATTI TEDRE is currently a Professor at the School of Computing, University of Eastern Finland. VOLUME 10, 2022