A Review of Naturalistic Decision-Making and Its Applications to the Future Military

Nearly 30 years have passed since Naturalistic Decision-Making (NDM) was introduced in 1989, and NDM has developed rapidly in recent years. The purpose of this paper is to discuss all background information relevant to the field of Decision-Making (DM) research. It also introduces the latest developments in this field and proposes its military applications. The first part is a review of NDM, including foundations of Naturalistic Decision Making, traditional analytical DM models, Recognition-Primed Decision (RPD) making methods and Schemata and Mental model. The second part is a review of the nature of expertise-experience, of Teamwork and Decision Errors in NDM, in which Teamwork consists of defining and Shared Mental Models (SMM), and Decision Errors consist of defining and teamwork decision-making error. The third part proposes the military application of NDM, including guiding military systems design and evaluation, teamwork decision-making error, optimizing training, supporting team collaboration.


I. INTRODUCTION
An important achievement of the Naturalistic Decision-Making community is the specification of the way common people actually reach decisions under real-world conditions (as opposed to the lab) [1], [2]. After thirty years since the emergence of NDM in 1989, it has been applied in several diverse areas, including health care, military services, aviation, command and control, complex engineering, and high-reliability industrial settings [3]. It can be efficiently employed under complex-risky emergencies, characterized by high level of uncertainty [4].
Orasanu and Connolly have identified eight factors often appearing in NDM settings, which are related to ill-structured problems, uncertain dynamic environments, shifting or illdefined or competing goals, action/feedback loops, time stress, high stakes, multiple players, organizational goals and norms [5]. The first three factors describe the fuzziness and uncertainty that a person may face when making decisions, which ''shitfing'' emphasizes that the goal of the decision changes depending on the dynamic environments and the The associate editor coordinating the review of this manuscript and approving it for publication was Jinming Wen .
team. The fourth factor describes that a decision is usually affected by multiple events. In order to achieve a goal, it is usually necessary to continuously adjust according to the feedback of the decision and finally achieve a satisfactory choice. The fifth and sixth factors indicate the situational characteristics of naturalistic decision-making, which ''high stakes'' emphasizes that making decisions is high-risk and wrong decisions can lead to fatal consequences. The last two factors describe that naturalistic decisions are made by individuals together and those goals are usually organizational goals rather than individual goals. A decision maker may utilize the naturalistic approach to reach decisions, in the case that some of the factors are present. In the presence of some of these factors, a decision maker can use naturalistic methods to make decisions [6]. Zsambok and Klein have summarized the following basic characteristics of NDM research [7], including the set of tasks and corresponding factors, the ones who are actively involved (experienced people who are reaching decisions, naive ones are excluded), the research target and goals (understanding the reasoning of experienced people under context-rich cases, not the way they should inference under a typical rational approach), and pay equal attention to awareness of the considered case, (do not offer all of your concentration only to the determination of the optimal option) [3]. Hoffman has noted that Naturalistic Decision Making is concerned with the way people make decisions in complex and uncertain real-war contexts. Such cases might require real-time decisions under conditions of emergency, with significant implications of errors [4]. After years of evolution, timely NDM systems have become capable to cope with the requirements of every operating system, and they are characterized by adaptivity and resilience. Moreover, they can handle unpredictable and dynamic cases [8]. Over the past 30 years, researchers on this field, have focused not only on decision making itself but also on how to deal with complex problems and how to improve current skills. Thus, researchers have been revising and supplementing the theory and models of Naturalistic Decision Making, including decision models, the role of Expertise, teamwork, and Decision error.

II. GENERAL THEORETICAL FOUNDATIONS-LITERATURE REVIEW
DM is an essential cognitive process that often occurs in every day life. According to Yates, a decision is a conclusion reached after consideration, related to a process that should produce the desired results [9]. Additionally, decision making is the action that proceeds the efforts towards the resolution of a specific problem. It aims to reach optimal action [9]. NDM researchers have created many models to explain this decision process. Before the development of the Naturalistic Decision-Making approaches, there was a wealth of research and development of traditional analytical decision-making models. The prescriptive and descriptive models are typical examples.
Prescriptive Analytical Decision-Making (PADEM) is an approach used in psychology [10]. According to this method, people should collect and analyze information and they should finally choose the best process among all potential ones. The validation of both strengths and weaknesses of every alternative is a basic requirement for the selection of the best solution. According to McDaniel who summarized the classic decision theory, people are comprehensively considering every potential in order to choose the option that offers the best case. This alternative was considered to be optimal, all other alternatives being suboptimal [11]. The essence of decision research is to validate the quality of DM, by comparing its outcome with those predicted by general statistical models. Classical approaches focus on how to reach optimal decisions which are mainly based on laboratory experiments, rather than on the way people actually decide in every day life [12]. Thus, the result is bound to fail to describe many DM processes adequately and accurately [7]. Due to the above, the research has evolved through the years and it has become more descriptive.
The rational choice model of decision-making is the most common example among a number of descriptive decisionmaking theories [13], [14]. Janis and Mann argued that high anxiety (a consequence of desperation or the result of shortage of time) is actually a sign of deliberate escape or ''hypervigilance'' [14]. It can lead to deficient quest and planning which results in an increased risk of disaster. They also suggested that decision-makers in a hypervigilant would be passively confronted with all the available information. Thus, they would fail to distinguish between relevant and irrelevant information [14]. Simon suggested that the case handling decisions are subject to limitations posed by ''bounded rationality'', which means organizations and their members find it difficult not only to gather all the information they may need, but even to process adequately the information they have [15]. Janis and Mann's research did not consider the ''expert's'' role in real-world cases. Under common settings, experts would make decisions based on past experience and information search strategies, rather than passively face all information [16]. The rational choice model recognizes high stress as a potential influence on decision-making. However, other NDM researchers argued that this DM method has some weaknesses when it is applied to war cases, characterized by high risk, uncertainty, time pressure, and dynamic conditions [13], [17], [18].
In general, traditional analytical decision-making, such as PADEM models, evaluates the pros and cons of every option and selects the optimal one [10]. However, other researchers have suggested that people do not always decide like this in reality. So, they developed a number of descriptive models. The Rational Choice Model (RACM) is the most common example, which emphasizes that high stress (especially time pressure) can affect the human decision-making process. Nevertheless, it has limitations under war conditions.

III. NATURALISTIC DECISION MAKING RECOGNITION-PRIMED DECISION MODEL A. THE NATURALISTIC DECISION MAKING
The 1988 USS Vincennes (VINC) incident, was a characteristic tragic event that gave food for thought. The VINC on a day with blue skies shot down due to a wrong decision, an Iranian passenger airplane, carrying 290 people [19]. The people on the cruiser mistook the Airbus A320 for an F-14 fighter jet and had believed they are being attacked. Nobody could give a rational explanation of why such a terrible wrong action was executed by the experienced commander of the ship. All of the public opinions were skeptical that such a terrible incident might be repeated in the future. Thus, they decided that significant initiatives should be taken, in order to design approaches that would facilitate people reaching proper decisions, especially when in dynamic cases of time-anxiety and uncertainty. Although the U.S. Navy had some knowledge of NDM decision-making models at that time, it was found that in real-world applications, these models could not explain how an experienced commander made decisions under the conditions of time pressure and uncertainty. In 1989, in Dayton, Ohio, USA, an invitationonly conference held to assemble DM researchers under war conditions. They extensively discussed the way people are inferencing, without examining and considering hypotheses related to principles of judgment [19]. This meeting revealed the limitations of research under laboratory conditions, that would make it impossible to fully extend laboratory research in-vivo environments [20]. All of the limitations showed that a novel DM methodology was required, which was named the Naturalistic Decision-Making approach [20]- [22].
Zsambok and Klein have concluded and suggested that the NDM is searching how qualified people (experts in the field, acting either alone or collectively, under dynamic and fast domains with high uncertainty) evaluate their situation. Additionally, Naturalistic Decision-Making examines how they reach decisions and how they undertake actions with consequences known to them and to their wider organizations [2]. Therefore, researchers should develop and employ the appropriate models that can satisfy the above definition. The RPD model (Fig. 1) is a well-cited and a well-established case [7].

B. THE RECOGNITION-PRIMED DECISION APPROACH
The RPD methodology ( Fig. 1) aims to model the reasoning process that exploits human experience in order to perform pattern recognition and reach decisions without comparing options in realistic environments [23]. The model is based on the actual observation and interview of the Fire Ground Commanders (FGC). Experimental data show that the probability of simultaneously comparing and evaluating two or more options in the decision-making process is less than 12%. Instead, FGC most often rely on their experience to directly identify typical situations and to identify an appropriate course of action applicable to that particular situation [23]. Systematic research that was conducted in the past, had proven that quite often FGCs did not use any comparative approaches in cases of difficult and challenging The employed approach concentrated in scene familiarity, specific corresponding indications, expectancies, potential targets, and suggested the Course of Processes (CPRO) for the critical situation.
In a real-world case, people are matching the situation with the patterns in their mind. If this process is successful, they are immediately executing the familiar course of action. This process is consistent with the actual situation, so many people recognized it. Therefore, the RPD model was used to describe the military decision making in fire ground command [24], in naval cases [25], in command and control [26], in aviation [27] and finally in battle planning [28]. One of the most important highlights of the RPD model is mental simulation. A commander evaluates an option without comparisons because the mental simulation is used to evaluate the CPRO and to imagine how it would play out in the current dynamic situation. The commander evaluates each potential option sequentially until the applicative one is found. This is a verification of Simon's point of view, searching for just any solution that works, instead of the optimal approach [29]. Making an overall assessment, the RPD model applies a reasoning that employs both intuitionistic and analytical procedures. Pattern machining represents intuition and mental simulation represents analysis [1]. Figure 2 shows the comparisons between the Recognition-Primed Decision model and the classic approaches.

C. SCHEMATA AND MENTAL MODEL
Lipshitz and Shaul argued that the RPD model is too simplified to fully represent the entire process of NDM [30]. They found several differences between novices and experts. First, experts search for information more efficiently because they know which messages need to be tracked among many messages. Second, experts are able to read situations more accurately because they can tell what is relevant and what is not. Third, experts' personal experience and the effectiveness of information collection are higher, so there are fewer wrong decisions. Fourth, experts have a better understanding of the relevant information and operating habits of friendly units. Fifth, experts tend to take other people's opinions into account when making decisions [30]. They suggested that this method does not provide enough explanations for their findings, and that it must change in order to comprise of both schematic and mental models [30]. Obviously, a schematic model provides a clear view of what course of action is expected, what is observed and recognized, and what is in our memory, in order to proceed to the inference process. The schema also provides a comprehensive view of missing data and how this problem can be resolved [12]. The inclusion of schema theory in the description of NDM explains why experts can take quick actions despite seeking more information in decision-making because they are guided by previous experience. A Mental model, however, is a labile entity that is developed and updated continuously following a life cycle, as conditions and situations change [30]. When a mental model is produced, the decision maker then forms his detailed view of the current problem, which helps the decision-maker set goals and expectations. The following Fig. 3 below illustrates the actual structure of the schematic-mental approach, as the core of the RPD model. It comprehensively depicts the flow, interdependences and correlations among its basic foundations, including situation, schema, mental model, and decision.

IV. THE ROLE OF EXPERTISE-EXPERIENCE IN THE NDM MODEL
NDM emphasizes the importance of decision-makers' knowledge and experience. Therefore, it draws on and promotes the scientific understanding of expertise [31]. The current understanding of professional knowledge generally has two important characteristics. First, expertise is the accumulated experience (through the years) on domain-specific knowledge. In other words, it is the knowledge-based view of expertise [32]. Second, psychological and physical adaptation to task constraints makes professional skills possible [33]. However, it is not only the length of experience that is important, but also the quality should be taken into consideration [23]. Shanteau has concluded and supported that expertise is the knowledge of people, who have proven through the years that they possess the required professional abilities, experience, and skills in order to guarantee optimal results in their field. He has introduced eleven specific mechanisms of expertise [34].
The application of expert knowledge in NDM is the most important factor to distinguish the good and the bad decision results. Domain experts who take the responsibility to make proper decisions have a clear view of the indications and contextual features regarding the situation they are dealing with and increased decision effectiveness [35]. Expert decision makers (EXDM), not only have a larger knowledge base, but they also organize its structure in a different way from the novice ones [32], [36]. They are performing pattern recognition, they can quickly gain a clear image of the case and they can select responses to situations of greater diversity and complexity, without consuming energy in forward-looking and in deliberative thinking [37]. EXDM seek diagnostic feedback to adjust their performance process and to improve their decisions [38]. They have better situation assessment and problem representations abilities than the novices, but not necessarily better general reasoning skills [39]. They automate the small steps and they focus on more complex aspects of performance [40]. Human experts reach their decisions by self-regulating and monitoring their process in order to make judgments about the consistency, reliability, and completeness of the information to increase the quality of the decision.

V. TEAMWORK IN NDM
In today's time pressure, heavy workload, dynamic, complex, and changing environment, decision-makers do not operate alone. This is because the complexity and difficulty of problems and environments have often completely exceeded one's ability to understand and make decisions [41]. Therefore, multiple experts come together as a ''collective'' to combine data coming from diverse sources, in order to enhance their effort towards the achievement of the goal. The difficult task that is still under research is to transform a team comprising of human experts to an expert team.

A. THE CONNOTATION OF EXPERT TEAMS
Salas defines the term ''group of experts'' as follows: ''It is a distinguishable set of two or more people who interact, dynamically, interdependently, and adaptively toward a common and valued goal/objective/mission, who have been assigned specific roles or functions to perform'' [42]. In this team, individual expertise must be coordinated so that expert decisions can be made at the team level [43]. Decision-making at the team level includes information collection or integration, action goal formulation, determination of the potential set of followed processes, solution selection and consequence evaluation [44]. Expert Teams (XTE) have the following characteristics: They develop shared mental models in order to form complementary explanations of environmental cues and implicitly coordinate their responses [45]; They learn, adapt, and shift strategies as needed in response to salient changes in the environment; Each XTE member is assigned clarified roles and responsibilities. This can improve decision making effectiveness; Members of expert teams share a well-defined, validated, and common vision; They maintain performance by following and applying a cycle or discipline of Pre-brief-performance-Debrief; XTE apply a deep team leadership pattern; and they follow a strong confidence and trust protocol, combined with collective mentality and team spirit; They cooperate and coordinate [46].

B. SHARED MENTAL MODELS
Over the past few decades, several NDM researchers have conducted studies on shared mental approaches (SMEA), and they all agree that teams tend to develop shared mental models (SMM). Cannon-Bowers suggest that the nature of how teams cope and adapt during stressful conditions can best be understood in terms of SMEA theory [44]. Stout et al. found that teams that planned effectively before tasks had better SMM than those that did not plan effectively and that teams planned more effectively performed better when workload increased [47], [48]. Druskat has introduced the elements of a shared mental model for teamwork as shown in Fig. 4 [49]. The experts in teams must share their mental models, in an effort to share a common image of the team and the running task. The connotation of sharing is mainly reflected in the following two aspects. First, team members can share mental models that hold the same representation of tasks, teams, and environments. Second, because team members can share a mental model, they distribute these representations across the team. Furthermore, Naturalistic Decision-Making researchers believe that SMM can lead to more active communication, which in turn improves the quality of team decisions.

VI. POTENTIAL DECISION ERRORS IN NDM
Through the decision procedure, experts are guided by their abilities and their experience in order to find a way to gather all important clues and to determine the final target and the optimal course of action. However, in practice, there is no guarantee that every decision will lead to the optimal result. Sometimes, we suffer from the tragic consequences of catastrophic errors. Thus, reduction of bad decisions' probability is one of the most important parts of the NDM research area.

A. DECISION-MAKING ERRORS
NDM research is of great help to cognitive activities related to decision-making errors. A systematic review process classified the errors in three major classes [50]. The first class comprised of errors that emerged due to the lack of experience. A subset of them was caused by the selection of a wrong analogue or by the inadequate use of an analogue. Errors of the second class were made because of the lack of information. The incomplete or inadequate mental simulation (which is really crucial) was the cause of the third class errors. Lipshitz has defined them as deviations from the typical-default decision process, which significantly increase the probability of bad results [51]. Endsley suggested that ''many human errors that are attributed to poor decision making actually involve problems with the situation awareness portion of the decision-making process as opposed to the choice of action portion of the process. Decision makers make the correct decision for their perception of the situation, but that perception is in error'' [52]. Orasanu et al. suggested that 19 out of 51 typical and frequent decision errors are due to the fact that the decision experts, fail to stop an approach which has already become unstabilized. These errors are characterized as ''plan continuation errors'' [53]. Although researchers have studied and they have analyzed the decisionmaking errors (DEME) that cause accidents, only a few of them have described the employed approach to examine the DEME that leads to accidents. It is important to notice that only when errors are detected in the decision-making process disasters can be avoided.

B. TEAMWORK DECISION-MAKING ERROR
As described earlier in this article, many decisions are made jointly by team experts in real circumstances, so teamwork decision-making errors need to be studied. Some NDM researchers believe that group decision making is often less effective than individual decision making [54]. There are three factors account for this: individual cognitive causes [55], process dysfunctions (organizational causes) and social dysfunctions (social causes) [56]. Process dysfunctions are caused by structural characteristics in the group setting. Social dysfunctions are caused by limitations inherent in the structure and form of meetings, which include socializing, conformity pressures, domination due to status imbalance, group think and so on. Therefore, any decision should be considered from three aspects: cognition (individual), organization (team/situation) and society (team/ situation).

VII. NDM'S FUTURE MILITARY APPLICATIONS
As mentioned in the previous content of this paper, NDM has been applied in many fields. This paper puts forward some thoughts on its future military applications.

A. GUIDING MILITARY SYSTEMS DESIGN AND EVALUATION
In tactical operations, the working conditions of military personnel may seriously affect their ability to work effectively. They are facing death in their everyday duties. Their reality includes bad or no sleep at all, terrible fatigue and poor physical condition, a lot of anxiety and fear of death. Thus, the need for human-computer interfaces that provide effective decision-making support to military command and control personnel is enormous. Thus, Naturalistic Decision-Making researchers should adjust the theories, models and research methods, to the human-computer interface of military information systems. The aim is to improve the support of the commanders' decision-making process. For instance, Bennett employs NDM methods in order to implement and validate a military command and control interface, and also to provide cases-examples for the tactical operations of the American Army, at battalion level [57]. Hall selected experienced officers (as participants) who evaluated the two alternative interfaces developed for military command and control. The obtained conclusion was that ecological interfaces are easier to learn and apply, and more effective than non-ecological ones [58].
However, the transformation to a large extent must be objective in order to ensure its effectiveness. It should not be influenced by the designer and it should not be system depended. NDM theory and research must be concretized to form precise and repeatable design guidance. As mentioned in the first chapter of this paper, NDM includes 8 factors, so the method itself is complex and changeable, which makes the design work difficult and challenging. In the area of testing and evaluation systematic, methods need to be developed to measure the human-system interface and system design usability, under the guidance of the corresponding theoretical background. Also due to the complexity of NDM, the usability test and evaluation of the system also has difficulties and challenges, because the establishment of a set of accurate usability test and evaluation system not only requires a lot of expert participation but also requires a lot of manpower and material resources, and a set of evaluation system cannot be applicable to all the usability test and evaluation of the system [59]. In addition, with the continuous development of information technology and equipment, human cognition and environment are constantly changing, and the existing interface design and evaluation methods may not be applicable, how to adjust will also be a challenge. Despite various difficulties, the application effect of NDM in the field of design is very significant, and it is also worth further study by researchers.

B. CREATING COMPUTATIONAL MODELS
Creating effective easy to use and navigate, software support tools capable of providing cognition aid to DM Experts under real-world circumstances have been the desired effect of NDM researchers. One of the core problems of Artificial Intelligence is to find the way to fit cognition processes of high level, in computational procedures and schemata (instead of force-fit) [8]. As technology continues to evolve, 5G/6G mobile applications, intelligent devices of the Internet of things, big data applications and cloud systems, emerging decentralized communication technologies keep emerging. Based on the above technologies, many applications will be derived. Many factors need to be considered when making decisions in a real-world environment, such as expert experience, teamwork, security, privacy, and confidentiality. The importance of this requirement is more prominent in military systems, which are characterized by high complexity, environmental uncertainty [60], teamwork [61] and multidimensional dynamic change [62].
It is both necessary and challenging to develop NDM-related models and methods into computable software models, and integrate the models into the information system, so as to help decision-makers recommend decision plans and improve the efficiency of decisions in the process of using the system. This task should be accomplished as computational methods often have to perform cognition at the meaning, knowledge, and context level in order to gain psychological credibility. Warwick first attempted to implement a computational case of a Naturalistic Decision Making cognitive model, aiming to develop a DM approach based on Recognition-Primed Decision theory [63]. This attempt makes RPD more than just a macro cognitive method, and it actually transforms the reasoning procedure to a more computationally performed search problem, where the search is performed in a group of pre-determined templates. However, NDM models are not mainly composed of input and output chains, so they are not fully implemented. Therefore, researchers have not made any more attempts on this basis. As NDM is a macroscopic cognitive process, the development and implementation of computational models are challenging. Warwick's attempt is a breakthrough. Military researchers should also put more effort into the computational models of NDM to develop models and software that can be used in the actual combat environment to support the improvement of decision-making process efficiency.

C. OPTIMIZING TRAINING
Experts are people who have comprehensive and authoritative knowledge or skills in a particular field, so experts must also invest a lot of time and accumulate a lot of experience in the relevant field. But due to the different types of tasks that have to be carried out by the military, not only experts in diverse specific fields, but also skillful working people are required. In general, because the cost problem of manpower VOLUME 8, 2020 and material resources, often does not need so many experts and skilled workers, but in an emergency situation often need to increase staff or decision makers (such as more and more in the Australian army reserve officers were asked to complete some tasks, despite the lack of experience but they need as ability to work effectively with full-time officials [12]), and the two are mutually contradictory. Therefore, it is of vital importance to quickly train a number of novices into journeyman. However, it is difficult and challenging for experts to summarize, extract and express their experience [64]. NDM researchers should explore how advanced knowledge and skills develop, in order to know how to acquire them, and to be transformed from novice to apprentice, to journeyman and finally to a human expert [65]. Ways to speed up learning for individuals and teams should be explored to speed up novice training [66], such as the research on the detection of mines [67]. We need empirical and reliable methods to identify individuals who are likely to be good mentors or to develop career paths for specific people who are likely to be expert mentors and domain experts.

D. SUPPORTING TEAM COLLABORATION
In the process of carrying out military tasks, the members of the team are either scattered or centralized, and they must complete the combat tasks through interaction and collaboration. Although Naturalistic Decision-Making researchers have collaborated with research teams, most studies have been conducted in laboratory conditions. However, it is difficult and challenging to obtain good applicability of relevant researches because the scene, environment, personnel state change, weapon state and other information observed in real-world conditions are different from those observed in laboratory conditions. Moreover, team collaboration and NDM decision making take place in a complex practical environment, so more attention should be paid to research on the execution of team cooperation tasks, in a realistic domain [68]. Besides, the training of team cooperation and the development of related products should be strengthened, and the relevant research results should be applied to the actual environment to help team members share situational awareness. The target is the improvement of collaborative operation efficiency.

VIII. CONCLUSIONS
As has been shown, NDM research has been developed for many years, and there are many examples of actual applications in the military field that would help commanders to make decisions under real-world conditions. This paper introduces Naturalistic Decision-Making related research, seen from the above point of view. The purpose is to provide rich and meaningful insights for researchers in the field of NDM, especially in the military field, and four application directions of NDM are proposed, including systems' design and validation, model integration, improvement of training skills, and teamwork. However, with the continuous development of the NDM field, more and more problems and challenges have emerged, because no one ''solution'' can solve all the problems. For example, how to further understand the importance of expertise in new fields, and how to explore new content and fields (such as intelligent analysis) [69], [70]. In some fields, there may be very novel decision-making problems, and we do not know how to solve them with NDM method, so relevant researches are very challenging. When exploring new fields, it is suggested to make breakthroughs by combining traditional decision-making research methods, NDM decisionmaking methods, big data [71], [72], neural networks, deep learning [73], active learning [74] and other methods.
Despite many difficulties and challenges, NDM has achieved twice the result with half the effort in various fields. Therefore, researchers need to invest more manpower and material resources to explore and make breakthroughs.