Player Satisfaction Model and its Implication to Cultural Change

Game refinement theory has been studied to derive a measurement of game sophistication. Recently, it has been developed as physics in mind, which may relate to the state of player’s feelings such as satisfaction and comfort in mind. This article explores a link between game refinement theory and reinforcement schedule. A notion of winning hardness to achieve the goal (<inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula>) of a gamified activity, relative to the reinforcement schedules (<inline-formula> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula>), is proposed through the physics in mind measures, which is utilized to quantify an activity’s enjoyment. By applying these new measurements to various gaming activities, the sense of player satisfaction and sophistication in game-playing is induced. Indicators for cultural changes and their implications to game-playing landscapes and experiences were established based on evidence from various well-known games. From such findings, the link between game refinement theory and reinforcement schedule may imply that classifying the games according to our mind’s psychological activities is vital in design decisions that largely influences people’s quality of life.


I. INTRODUCTION
Games had been used in many areas (such as learning [1] and business [2]) to promote entertainment. Among the mechanisms used to increase enjoyment involves usage of game elements [2], competition [1], and psycho-physiology [3]. Since rewards played an integral part in gamified interventions [4], the underlying mechanisms of in-game rewards are relatively limited and understudied. Rewards have various forms, where popular examples are high score, experience points, feedback messages, and game-playing mechanic [4].
The concept of reward is linked with a reinforcement learning theory, advocated by [5], which is a doctrine of understanding and correcting human behavior [6]. In its most basic form, the reinforcement learning theory refers to the positive or negative consequences (remuneration or punishment) of an action [7], [8]. The goal is to determine a behavior recurrence which constituted from an agent's probability response to stimuli. Based on many experiments conducted by Skinner on behavioral stimulation [8], variable-ratio (VR) The associate editor coordinating the review of this manuscript and approving it for publication was Yongqiang Cheng . schedules (rewards are given after a random number of correct responses) was found to have the highest response rate, which shows repetitive and straightforward rewards for doing one thing is not the best way to elicit the expected behavior. The effectiveness of such a schedule can be improved if the reward is randomly changed after several actions.
Game designers use this principle to create the illusions of implicit motivation for players to extend the playtime. Such a reward schedule model will encourage players to obtain rewards and continuously strengthen their attractiveness stimuli towards the games. Measuring the attractiveness of the game by identifying the underlying mechanisms of motion in mind concept had been proposed [9], which is derived from the game refinement (GR) theory [10]. It is a crucial evaluation standard which showed to be effective in many different game fields. The main point of game refinement is not winning games and beating opponents but concerned with the game sophistication and entertainment of the target game perceived by the players. The notion of motion in mind involves identifying the players' enjoyment and engagement, where the underlying characteristic of games are analyzed to improve its affinity [9]. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ In this article, a reinforcement paradigm based on the variable ratio (VR) schedule is adopted to establish the link between the reinforcement schedule and game refinement theory. The GR theory is also utilized as the methodology for assessing games, where a new game progress model and physics in mind measures based on the VR schedule were proposed. Then, the link between the underlying game mechanisms relative to various human culture was established.

II. PLAYER SATISFACTION MODEL IN GAMES
A. GAME PROGRESS MODEL A game progress model by [11] involves modeling the amount of solved uncertainty of the game as a function of x(t) based on an increasing function of time t. A realistic formulation of game progress with the known outcome is given as (1). The parameter n (1 ≤ n ∈ R) is the number of possible options and x(t) is normalized within the range of 0 ≤ x(t) ≤ 1. Deriving x(t) twice at t ∈ [0, T ], given by (2), it indicates the change velocity or rate of acquired information (acceleration) of the solved uncertainty of a game. Then, a measure of game refinement (GR) is (root square form) given by (3), which represents the metric for game sophistication.
The GR measure has been adopted and verified in various types of games, as demonstrated by previous studies [12]- [14]. For the board and scoring games, the GR measure is determined by (4) using the model of move candidate selection and scoring rate [9]. Here, B and G stands for average branching factor and average goals, respectively. Meanwhile, D is the game length (total number of plies), and T is the total points or goals. These respective variables were collected from the average of the total number of play-testing experiments. The sophistication of games converges to almost similar sense of thrill (or noble uncertainty [15]) of GR ∈ [0.07, 0.08] ( Table 1).

B. VARIABLE RATIO SCHEDULE (N) AND WINNING HARDNESS (m) IN GAMES
In VR schedules, the parameter N shows the average reward frequency, where 1 < N ∈ R. In this study, winning a game corresponds to obtaining a reward, then it implies the game length, which is D in board games (total number of plies) and T in scoring games (total points or goals). Hence, N = D or N = T , implying a general form of reward frequency of the game's winning rate. Based on such a notion, the winning rate v and winning hardness m is defined by (5).
Analogical links between motions in physics and motions in mind had been previously established based on the notions of winning rate (or velocity) v and winning hardness m [9]. The correspondence between the physics model and the game progress models is established as in Table 2. Such correspondence enables the measures of physics in mind in various games, specifically on three quantities: potential energy, momentum, and force. The potential energy (E p ) in the game is defined as the game playing potential or the expected game information required to finish a game [9], given by (6). Meanwhile, momentum ( p) in the game refers to the competitive balance of a game, which involves the degree of challenge needed (m) and effort given (v) to drive the game progression [9], given by (7).

D. FORCE IN MIND AND PLAYER SATISFACTION
Arnold Toynbee, a British historian, had asserted that ''the supreme accomplishment is to blur the line between work and play.'' Such assertion can be found when F = p demonstrated when having a = 1 N . In such a situation, F corresponds to the player's effort to move in the game (work) while p corresponds to fascinating or seesaw in the game (play). Hence, sophisticated games had accomplished such a notion (F = p), which blurred the boundary of work and play. Previous work by [9] had defined the F as the player's strength to move a game or ability in general, where a is the growth rate of ''flow'' experience of the player in the game (since a = F m , then F is the ability and m is challenge [19]). In this study, a = 1 N can be regarded as the sense of gravity in people's minds, where it is the source of cultural tendencies of people's minds in game-playing reflected at a specific time/era. Hence, the measure of F is given by (8).
Sophisticated board games such as Mah Jong, Chess, Shogi, and Go have distinctive origins and represent various developments of cultures, as given in Table 3 and depicted in Figure 1.  Based on the historical establishment of various board games, the Go game is the oldest (established in the 5 th century), followed by Shogi, Chess, and Mah Jong. At the time of Go game was established, the gravity of people's mind favors conservative play and long-term gain (frequency of reward is low; N 200). Meanwhile, Chess and Shogi both closely established during people's minds that gravitate towards more aggressive play and medium-term gain (medium reward frequency; N 100). Meanwhile, Mah Jong established relatively recent (20 th century), where the gravity of people's mind favors high reward frequency (N 50) and increasingly fast-paced (higher p).
Concerning the GR measure, the convergence of the approximately similar values (or GR zone), can be explained by the sense of excitement and thrills given by the games at the respective time/era of their establishments.
Conjecture 1 (Player satisfaction model): People would feel the sense of satisfaction, sophistication, and fairness in a game iff p = F. This situation also implies N = 1 a , where 'a' changes in history and symbolizes the cultural drive at the time, which is equivalent to the magnitude of gravity in people's minds.

III. PHYSICS IN MIND AND CULTURAL CHANGE
The definition of play, as given by [27], [28], is the essential activity for striving societies and provided the necessary conditions for the cultivation of culture. Like the development of a civilization, a play requires structure and participants willing to create within specific limits. Starting with Plato, [27] traces such notion, in the contribution of ''Homo Ludens,'' (or ''Man the player'') through Medieval Times, the Renaissance, and into the modern civilization. The concept of culture and play evolves side-by-side as a civilizing function that ultimately influences people's value of life. To demonstrate such values, games are analyzed from the perspective of reward frequency N using the notion of gravity in mind (a) to identify relevant phases of cultural changes.

A. FIRST PHASE FROM GO EVOLUTION
Observing from the oldest board game, the Go board has the longest history [20], which originated more than 4000 years ago and a history of the development of around 2500 years (Table 4). Its development had been observed to change from N 60 to N 200, with a = 0.01 to a = 0.004, respectively ( Figure 2). It can be inferred that people strive towards conservative activities where the culture changes from short-term to long-term reward frequency. Such an environment fosters increasingly stable conditions (low p and E p ) and knowledge-driven (based on its increasing B; more options per move).
to the modern western Chess, approximately 1600 years ago [23], [24]. Its development is observed to change from N 200 to N 100, 1 with a = 0.05 and a = 0.01, respectively ( Figure 3). During this time, the evolutionary directions of Chess are in contrast to the Go, where the culture promotes medium-term reward frequency (moderate p and E p ), albeit knowledge is valued (small increase in B; more options per move).

C. THIRD PHASE FROM MAH JONG EVOLUTION
Mah Jong had originated about 600 years ago, with around 50 years of development (Table 6), which is demonstrated by 1 N = 100 indicates that computer self-play experiments have a longer game length compared to human data (Table 3) due to the lack of resignation.  [11], [15].  [15].
N ∈ [30,50] and the a ∈ [0.02, 0.05] that roughly stays within a relatively similar value (Figure 4). This situation showed that people's culture favors high reward frequency (N < 50), where fast-paced activities (high p and E p ) were found more attractive, and they are more engaged when the game is less complicated and leave rooms for uncertainty (small B; fewer options per move).

D. EVOLUTION OF SPORTS GAMES
Meanwhile, popular sports games such as Basketball and Soccer were also analyzed to observe the evolution of a, where minor incremental changes were observed in both games. The data from the world's league games of Basketball and Soccer games were collected (NBA 2 and FIFA 3 ), where G is the average shots (or scores), and T is the average total shots attempts (or tries) (exception for the blocked shots per game in Basketball).
The data of Basketball and Soccer are given in Table 7. It can be observed that sports games took different directions compared to board games ( Figure 5). Both games showed contradicting trends, where Basketball becomes very difficult to gain a reward (high N and m) and demands skillful play (resembling mind sports, i.e., board games). Meanwhile, Soccer is becoming more stochastic (high p) and fast-paced (low N ), matching the gravity felt by people's minds in modern times. This condition is interesting since Soccer had a  relatively long history (2000 years of history 4 ), albeit widely accepted as a contemporary sport worldwide.
Another example of contemporary sports such as Table Tennis, was also observed to change a when its rule changes from 21-point system (N 200) to 11-point system (N 100) [16]. Such evolution of gravity (a) may still be on-going in many modern games (such as video games), where a increases closer to the addiction zones (E p = F), which logically sound with the increasingly aggressive game markets.

E. EVOLUTION OF ACTION VIDEO GAMES
Since the 1980s, video gaming has become a popular form of entertainment and a part of modern popular culture in most parts of the world. In video games, players' experiences and feelings will be sought out by immediate stimulation to their eyes and ears [18]. A video game sub-genre called action games is a video game that emphasizes physical challenges, including hand-eye coordination and reaction time. The genre includes a large variety of sub-genres, such as fighting games, shooter games, and platform games. Based on the previous 4 https://www.footballhistory.org/ study conducted on fighting video games [31], [32], T and G is the average actual attacks and the effective attacks, respectively. The results are given in Table 8, which reflected the development of action games. By observing and analyzing the fighting games released between 1985 to 2017 (see Table 8 and Figure 6), it can be deduced that modern people's minds gravitate towards increasingly fast-paced (increasing p) and a more frequent reward gains. This situation justified that fighting games are focusing on increasing entertainment where the game consists of high tension or pace onto the players (e.g., high moving speed), rather than merely competing. Also, the decline of T and increasing G from 1985 to 2017 showed that fighting games are trying to let players feel playful longer within moderate game length [32].

IV. DISCUSSION AND IMPLICATIONS A. DEVELOPMENT AND CURRENT TRENDS OF GAME AND HISTORY
Since the value of a can be regarded the measure of gravity in people's mind (a), it serves as an indicator to the core VOLUME 8, 2020 culture of people of a specific era or time, as suggested by [27], [28]. Hence, analyzing the changes of a provides insights into people's cultural tendency, based on their game playing experience and comfort.
According to the Merriam-Webster definition of game, 5 one of the tacit meaning of the term is used in hunting, referring to wild animals hunted for sport or food. Based on the historical development of Go game (B.C. 23 th to A.D. 5 th ), the reduction of a demonstrates the transition of people from nomadic culture (hunter-gatherer) into sedentary culture (agriculture), where people search for a more stable lifestyle (first phase; Section III-A). Suppose that the N is approximated as the day with the reward a year, then the hunting processes is equivalent with the reward requiring short-term work (N ∈ [50, 100]), while agriculture (e.g., crops like wheat and rice) requires long-term work (N ∈ [150, 200]).
Another direction of history is the transition from agriculture to a more exciting lifestyle (leading to the industrial revolution and free capitalism), where a increases and N decreases (second phase; Section III-B). Such historical development is related to the Chess game (A.D. 4 th to A.D. 16 th ), where war, conquest, strategy, and tactics, played an essential role in the early people's lifestyles (only the elite played such board games). The people's culture then shifted towards capitalism (efficient transportation such as horse-powered transport and railroads, and voluntary goods trading) and industrialization (mass production and machinery usage), allowed more frequent reward gains with medium-to short-term work ( from  N 200 into N 100).
The third phase of the evolution of Mah Jong is the period of transition in history, where both the a increases further and decreases again (N decreases and increases). Such historical development implies changes in people's culture towards fast-paced and highly accessible goods via inventions (such as mechanization since 13 th century and computing devices in early 19 th century) and modernization (first and second industrial revolutions). With such changes, the reward frequency significantly improved (N 30) and stabilizes with an increase of urbanization (N 50).
Based on the historical development of a for the board games along the period of their development (Figure 7), a symmetry-like trend was observed, bordering at about 4 th to 5 th century. Such a trend overlaps between the Go and Chess game's development, where a showed opposing trends between Go and Chess. Such a symmetrical border implies the turning point of the game's development, which is demonstrated by the first phase (Section III-A) and the second phase (Section III-B) of the game's evolution. It would also represent the border between competitive and mastery activities.
Conjecture 2 (Border of Competitive and Mastery): The historical development trend of popular board games showed that N 200 is the border between competitive and mastery, where it possessed a low 'a' (leisurely play) and 5 https://www.merriam-webster.com/dictionary/game extremely low E p (knowledge-driven or skill-based). Beyond such a border gives F > p, implying that players' ability to overcome the competitiveness of a game and experience rarely becomes rewarding. Such a situation also is equivalent to a turning point of gravity in people's minds, at about a 0.005.

B. GAME-PLAYING LANDSCAPES
Various games considered in this study constitute three distinct sports landscapes: mind (or m-sports; e.g., board game or abstract games such as Go, Shogi, and Chess), physical (or p-sports; e.g., Basketball, Soccer, Table Tennis, and Badminton), and electronic (or e-sports; e.g., DoTA and action games). A rough approximation of the ranges of the respective three landscapes of sports was depicted in Figure 8. It can be observed that those sports developed within an overlapping range of about N ∈ [20,200]. However, their distinction was based on their individual development. P-sports have been observed covering most of the ranges of the N , although sampled from a very specific game. However, this showed that physical-based activities had undergone a stable development since it is one of the oldest forms of play in existence alongside human civilization. As such, p-sports have been observed to provide a multitude of different game playing experiences, from luck-based play (e.g., Soccer), fair play (e.g., Table tennis and Badminton), to a skill-based play (e.g., Basketball). However, the implications of p-sports development remain too broad to be adequately determined and classified.
Meanwhile, m-sports have been observed to overlap with half the ranges of N of the p-sports. However, m-sports converge at about N ∈ [60, 200], where the game provides more specific game playing experiences where little to very knowledge-driven games are valued. Also, e-sports have been observed to change rules within a short time (less than ten years or less) while maintaining a range of values that are situated as the middle ground between those observed in the m-sports and p-sports. The game playing experiences of the e-sports also specialized in balancing skill and chance elements in the game. Interestingly, the direction of the three sports landscapes seems to be closing the gap between the E p = F and p = F, where the order of the gap size reduces from the e-sports, followed by the m-sports, and then the p-sports.
With reference to Figure 8, a summary of the possible landscape of known games, is given in Table 9. For the region of N ≤ 20, the reward's frequency is very high and requires low ability, motivation, and effort, which drives the player's curiosity. Such a region implies an activity that exhibits reinforcement effects that closely resemble the continuous reinforcement schedule that posses high reinforcement extinction effect [33], which could potentially lead to addiction. Meanwhile, a region of 20 < N ≤ 200 involves a rapid change of E p and p, which relates to competitive activities where it is often rewarding and sometimes motivating, which is dependent on the player's ability (or skill). The region beyond N > 220 is where both the ability and effort are high (mastery) where the activity becomes habitual and challenging to be motivating.

V. CONCLUSION
Game is a learning process where players learn and adapt to grasp the rules of the game. Similarly, reinforcement schedules, which were explored by Ferster and Skinner [34], had been widely used in the learning environment. Based on such circumstances, game settings become essential factors that affect the player's experience [9]. Game refinement theory and its application in various games have recently shown significant effects for evaluating games' entertainment--a successful bridge between learning and the player's engagement.
The variable-ratio of the reinforcement schedule, specifically the reward frequency variable (N ), defines the unexpectedness of achieving a reward (or score), which allowed the establishment of such a link, where various physics in mind measures were formulated. Potential energy defined the expected game information required to finish a game, implying that high energy would require less effort to play. Meanwhile, momentum defines the competitive balance of a game between effort and challenge to drive game progress, where high momentum makes a game exciting and fair (or having more frequent seesaw turnover). Force in the game defines the player's strength to move a game or ability in general.
The player satisfaction model given by a = 1 N is identified as the magnitude of gravity in people's mind when p = F. In addition, a was demonstrated to represent changes in history where it serves as an indicator for the cultural drives that is equivalent to the feeling of gravity in mind of people at different time. Game development trends also indicate the border between competitive and mastery in conducting tasks, which suggests a direction towards the higher value of p and E p , while smaller N (such as N ≤ 20). Such a condition would be inferred that high p and E p with N ≤ 2 would relate to a situation that induces high curiosity (motivated effort) to addiction.
The measures of physics in mind and player satisfaction model successfully established the relationship between game-playing and rewarding experiences, albeit in a minimal perspective. Potential future works may include exploring the dynamics of challenge and its relations to addiction. Such a measure can also be incorporated to improve game playing experience where a timely rewards schedule can be catered according to the psychological needs of specific players and their playing behavior (related to the field of player modeling).
KANG XIAOHAN received the M.Sc. degree in game informatics from the Japan Advanced Institute of Science and Technology (JAIST), Nomi, Japan, in 2020, where she is currently pursuing the Ph.D. degree with the Graduate School of Advanced Science and Technology. Her current works focus on establishing the link between psychology and game refinement theory, under the supervision of Prof. Hiroyuki Iida in the area of entertainment technology. Her research interests include distributed game refinement theory in psychology area, reinforcement learning, gamification in education, physics in games, mathematical model, data science in gaming, reward function in deep reinforcement learning, and its relation to addiction. His specializations are artificial intelligence techniques, game informatics, evolutionary computing and algorithms, and decision support systems. His work focuses specifically on the methods for optimization and game informatics in fields of operation research and entertainment technology. His research interests include but are not limited to manufacturing systems, artificial intelligence techniques, game analysis and informatics, search algorithms, optimization techniques, advancement in scheduling and planning, and machine learning methods.
HIROYUKI IIDA received the Ph.D. degree in heuristic theories on game-tree search from the Tokyo University of Agriculture and Technology, Fuchu, in 1994. He is currently a Japanese Computer Scientist and a Computer Games Researcher with a focus on game refinement theory, opponent model search, and computer Shogi. He is also the Trustee and the Vice President of Educational and Student Affairs with the Japan Advanced Institute of Science and Technology (JAIST), the Director of the Global Communication Center and the Research Center for Entertainment Science, and the Head of the Iida Laboratory. He is also a Professional 7-dan Shogi Player and a Coauthor of the Shogi Program Tacos, the four times Gold Medal Winner at Computer Olympiads. He was affiliated with Shizuoka University, Hamamatsu. He was a Guest Researcher with Maastricht University. His research interests include artificial intelligence, game informatics, game theory, mathematical model, search algorithm, game-refinement theory, game tree search, and entertainment science. He is a member of the Board of the ICGA as a Secretary-Treasurer and a Section Editor of ICGA Journal.