Integrated Augmented and Virtual Reality Technologies for Realistic Fire Drill Training

In this article, we propose a novel fire drill training system designed specifically to integrate augmented reality (AR) and virtual reality (VR) technologies into a single head-mounted display device to provide realistic as well as safe and diverse experiences. Applying hybrid AR/VR technologies in fire drill training may be beneficial because they can overcome limitations such as space–time constraints, risk factors, training costs, and difficulties in real environments. The proposed system can improve training effectiveness by transforming arbitrary real spaces into real-time, realistic virtual fire situations, and by interacting with tangible training props. Moreover, the system can create intelligent and realistic fire effects in AR by estimating not only the object type but also its physical properties. Our user studies demonstrated the potential of integrated AR/VR for improving training and education.

In this article, we propose a novel fire drill training system designed specifically to integrate augmented reality (AR) and virtual reality (VR) technologies into a single head-mounted display device to provide realistic as well as safe and diverse experiences.Applying hybrid AR/VR technologies in fire drill training may be beneficial because they can overcome limitations such as space-time constraints, risk factors, training costs, and difficulties in real environments.The proposed system can improve training effectiveness by transforming arbitrary real spaces into real-time, realistic virtual fire situations, and by interacting with tangible training props.Moreover, the system can create intelligent and realistic fire effects in AR by estimating not only the object type but also its physical properties.Our user studies demonstrated the potential of integrated AR/VR for improving training and education.
T he demand for contactless services surged during the prolonged COVID-19 pandemic.Extended reality (XR) technology, a key element of contactless services, has attracted considerable research attention.The demand for XR is expanding into everyday life and business areas, with applications in various fields such as education, training, healthcare, and defense.XR technology is a concept that encompasses both AR and VR.AR is based on the real world and enables realistic and natural interaction, but has spatial limitations.VR is based on the virtual world, providing high immersion and no restrictions on various spaces, but it is different from the real experience and difficult to experience for a long time.The important thing is that if these two technologies are used in a hybrid form, they can complement each other's shortcomings and show synergistic effects.
Many training applications using AR/VR or XR technologies have been developed because they can overcome problems such as spatial constraints, risk factors, training cost, and difficulties in real-life experiences. 1 However, most fire drill training research works tend to only support indirect forms of experience that cannot dynamically reproduce real-life fire situations, primarily due to the use of conventional controllers and predefined simple scenarios.AR can provide users with a realistic fire response experience by overlaying virtual fire elements onto real-life spaces, enabling trainees to acquire proper skills using real fire extinguishers.VR is more suitable for evacuation training because it can overcome spatial and safety-related limitations.In a virtual environment, large-scale evacuation training can be conducted without the limitation of physical space.
In this article, we propose a system for combining AR and VR experiences for fire drill training.A single training story is composed by linking two scenarios (AR fire suppression training and VR fire evacuation training) using a single head-mounted display (HMD), as shown in Figure 1.To augment realistic fires in a physical space, objects that can catch fire should be recognized in the real environment.Existing research has primarily concentrated on real-time object recognition for detecting human-object interactions.However, for the purpose of fire drill training in AR, we propose a novel approach to generate intelligent and lifelike fire effects.This approach involves estimating parameters such as combustibility, type, size, mass, and material in order to create realistic fire effects.

Integration of AR and VR Experiences
AR and VR integration blurs the boundaries between reality and virtual spaces and is the underlying technology for the highly anticipated metaverse.The compatibility of AR and VR technologies is critical in these techniques.Reality-based AR and virtual-based VR have distinct applications, strengths, and weaknesses.However, both technologies share the same objective of extending reality.Studies have demonstrated the effectiveness of integrating these technologies. 2Such mixed platforms can provide a significant advantage by enabling users to remotely perform tasks simultaneously.Although accessing and representing a single object simultaneously has limitations such as ownership features and interaction interface challenges, various studies have been conducted on collaborative mechanisms to solve these limitations. 3Sharing and collaborating each other's information and analyzing it within a mixed environment is a critical topic of research. 4Most AR and VR studies focus on remote interactions across multiple devices, with only a few XR devices supporting both AR and VR on a single device.Therefore, limited studies have been conducted on using AR and VR on a single device.Field et al. proposed a novel concept using Microsoft's Hololens to divide one space into a VR space (warpzone) and another space into an AR space (deskzone) to enable users to have distinct environmental experiences depending on the space. 5Another research team introduced a fun application that applied AR (third-person view) and VR (first-person view) to drones that navigate through a virtual maze. 6When combining AR and VR, the visual transition technique between reality and the virtual world plays a vital role.Fabian et al. explored four visual transition techniques for switching between AR and VR and found that these transitions effectively reduce the sense of isolation in VR. 7 In the studies, AR and VR were combined and only virtual objects were used for interactions.In our previous study, we proposed a method that could provide users with a realistic virtual experience by combining the real environment and virtual environment based on reality. 8In this study, we extend this concept and introduce a novel method to create a highly realistic virtual environment through the recognition and analysis of the detailed information of real objects in real time.

Synchronization of the Real and Virtual Worlds
3-D reconstruction is a technology that represents 3-D space with continuous 2-D images.Studies have been conducted on modeling virtual worlds using this 3-D reconstruction technology.Guo et al. modeled virtual environments and objects identical to reality and synchronized them with reality to implement a mixed reality (MR) office, presenting a technique to effectively utilize office space. 9With the development of the metaverse, numerous studies based on deep learning have been conducted on recognizing and interacting with objects in MR.Yalda et al. revealed that AR and AI can be applied to various industries and systematically reviewed research integrating AR, MR, and deep learning for object detection. 10Accurate alignment between virtual and real-world objects is essential when implementing 3-D reconstruction.In real-time systems, users are particularly sensitive to any incongruity in the movement of virtual objects that deviates from the behavior of real-world objects.Their study proposed a real-time 3-D reconstruction method that leverages physical information about objects and their surrounding space to improve the accuracy of object recognition and minimize such incongruities.
In VR, input systems such as controllers are typically used for interaction.By contrast, AR interactions are typically based on image recognition rather than physical controllers. 11Integrating these two technologies provides the advantage of freely using both methods.Furthermore, synchronizing virtual and real environments allows actions or information from one environment to influence the other environment.For example, virtual fire can simulate a sensation of burning, or firing a virtual bullet can simulate a ricochet effect when blocked by real objects, sharing various effects. 12,13They proposed a method that enables real object interactions to influence the virtual environment and virtual objects.

AR/VR-Based Training
AR/VR technologies are being applied in almost all fields, including entertainment, advertising, education, training, and therapy.In particular, numerous studies have been conducted to apply these technologies in training.Training is a teaching method that involves repeating a certain learning process to acquire skills or enhance understanding.In some fields, studies have been conducted to address the cost and time difficulties associated with repetition because of resource constraints for training. 14Gasteiger et al. proposed training using AR technology to overcome barriers in medical professional education with limited resources. 15The proposed AR training enabled visualization, interactive learning, skill enhancement, and repetitive practice in a safe learning environment, which improved the training outcomes and satisfaction of learners.We introduce a novel fire drill training system that combines AR and VR to enhance the effectiveness of fire drill training.

The Training System Structure and Scenarios
The structure of our system is shown in Figure 2. The dotted boxes represent the features directly implemented for this research, while the solid boxes represent the open-source tools and hardware used.We used the VIVE Pro HMD to implement AR and VR functions in a single device.Although this device is dedicated to VR, AR functions can be implemented through the built-in stereo camera.SRWorks is an SDK for using the stereo camera in the VIVE Pro, providing features such as depth perception and rough object recognition.The AI models recognize more detailed information such as volume and mass from the roughly recognized objects.The control manager supports interactive features for fire training such as

FEATURE ARTICLE
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
syncing a real fire extinguisher with a virtual one, AR/VR transition, and locomotion.Training on the system is categorized into two parts, namely fire drills in real spaces and evacuation drills in virtual spaces (see Figure 1).Fire drill training in the real part includes immersive fire drill training that recognizes the space in detail and fire drill training to suppress virtual fires that occur in this part.Realistic fire effects can be generated by collecting various properties about the objects in the physical space, such as their type, size, mass, shape, orientation, location, material, and other relevant attributes [see Figure 1(a) and (b)].Special effects suitable for each physical property are defined and implemented using the identified objects to generate realistic virtual fires [see Figure 1(c)].Users can suppress virtual fires using a real extinguisher prototype that is perfectly synchronized with the virtual extinguisher [Figure 1(d)].After completing the training in the real space, a portal is created to connect to the virtual world training part.Using this portal, users can move to the virtual space and perform fire evacuation training [see Figure 1(e)].During the virtual evacuation training, virtual characters provide evacuation guidelines; users should follow the training to evacuate safely [see Figure 1(f)].If the user escapes the building by following the evacuation guidelines, the mission is successful.However, if the user repeatedly disregards the guidelines, then they fail in the mission.This system provides a comprehensive and realistic training experience that bridges the real and the virtual worlds.

Combining the Real and Virtual Spaces
In this study, we introduce a transition technology that enables users to freely move between real and virtual spaces and propose its application method.To utilize continuous transition technology, equipment that can run both AR and VR without altering the HMD is required.The performance of the built-in camera in the HMD used in this research is not optimal, but it is meant as a preliminary study for AR and VR integration.The transition between AR and VR provides a natural learning experience.As the training smoothly transitions from AR to VR, learners can better understand and remember the content, which may assist them.Also, the continuous use of AR and VR can enhance the user's sense of immersion.The transition between AR and VR is also effective for efficient time management.All training takes place on one platform, so users can save time spent on switching environments or setting up new equipment.Finally, it can induce active participation.If the process of transitioning from AR to VR is directed in a way like a portal, users will feel as if they are actively changing the environment.This increases motivation for learning and has the effect of inducing user engagement and concentration.We used a portal as a means of transitioning between reality and the virtual world, and implemented a virtual door as the form of the portal as shown in Figure 3. Users can connect and move between the two worlds through this virtual door in an intuitive manner.This method produces a smooth and immersive experience by using Unity 3D engine's stencil technology to mask unwanted pixels and allow users to see only desired parts.

Creating a Realistic AR Training Environment for a Fire Drill
To automatically augment realistic virtual fires in a real space, objects in the real environment that can catch fire should be identified.Existing research on object recognition in AR has primarily focused on accurately identifying objects in real space and facilitating user-object interactions.However, simply overlaying predefined animations or repeatedly displaying the same special effects through recognized objects can result in a lack of realism and an incongruity with the physical world.To address this issue, we propose a system for implementing intelligent and realistic special effects in AR environments that estimates not only the object type but also its physical properties.In our system, AI models were used to estimate physical properties and intuitively generate appropriate special effects (see Figure 4).The estimation of physical properties is performed by SRWorks SDK AI model, Mobile-Net-v2 model, and a self-made mass estimation model sequentially.First, the SRWorks AI Model was used to acquire information such as the type, location, and orientation of large objects and reconstruct the recognized objects in 3-D.This module can recognize approximately 20 objects commonly used in general rooms, such as chairs and tables, and can distinguish the type of objects in large categories and identify their orientation and location.The reconstructed information was then used to estimate the volume of the target object.Second, the MobileNet-v2 model was used to estimate the object's detailed features and materials.MobileNet-v2 is a model that effectively reduces computational load by utilizing depthwise convolution.To optimize the learning process for this article, we have employed transfer learning to minimize the required data and associated costs. 16Specifically, we used a fixed feature extractor technique to remove only the final classifier layer from a pretrained model and froze the weights of the remaining layers.We then added a classifier for the new dataset and performed classification.Identifying materials is a critical factor in creating suitable fires, because various physical properties such as combustibility and object mass vary with the material.Finally, a learning model that can estimate mass after inputting the object size information through a multiple linear regression technique was created for each object.Among various pretrained mass estimation models, the model corresponding to the class information obtained from the image recognition network was selected.The object's mass can be estimated when its size information is passed to the model.Once an object is recognized by the AI, the flame is generated at the global coordinates of the recognized object in 3-D space.After the flame has been created, the AI model's role is complete.Figure 4 displays the process of recognizing various physical properties of furniture in this system and implementing realistic special effects.In this figure, the process of recognizing a fabric chair and a wooden desk drawer can be compared and verified.When set on fire, a fabric chair tends to propagate flames more rapidly, has a shorter burning time, and produces smoke with a darker hue.In contrast, a wooden desk drawer tends to burn more slowly, has a longer lifespan, and produces smoke with a brighter color compared to a fabric chair fire.Accurately identifying an object's physical properties enables the system to determine whether it is combustible and how it will burn.The object's size was utilized as an input to determine the intensity of combustion phenomena, including flame and smoke size.The object's mass serves as a measure of fuel quantity for combustion phenomena and affects the overall combustion duration.To implement the real-time creation and control of special effects, Unity's particle system was utilized.By augmenting special effects to the 3-D location of the recognized object, the system can simulate natural-looking flames.

Fire Extinguisher Training in AR
In virtual training, realistic scenes are essential, but the effectiveness of training is significantly improved by simulating behaviors that closely resemble real-life experiences.Therefore, we developed a prototype by modifying an actual extinguisher used in real-world training (see Figure 5).In this prototype, a water extinguisher was used for tangible interaction in real fire training, and it was developed to be fully synchronized with the virtual extinguisher.First, we used a VIVE Tracker to synchronize the position and orientation of the virtual extinguisher.Next, we selected a gyroscope sensor to synchronize the orientation of the extinguisher hose.The tethered hose on the extinguisher

FEATURE ARTICLE
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
body allows for synchronization based solely on directional information.With a lightweight sensor, this setup provides a highly realistic experience akin to using an actual fire extinguisher.We attached a small button to the extinguisher handle to release virtual powder.This measure enables users to tightly grip the handle and shoot the virtual powder at the desired location.Finally, we used an Arduino UNO Wi-Fi R2, which supports both wired and wireless connections, to control these sensors.Virtual flames can be extinguished by shooting virtual powder through this extinguisher.The size of the flames decreases with the duration of contact with the virtual powder and increases when left unattended over time.Each virtual flame is assigned different values based on the physical properties of the burning object.Different materials exhibit varying burn rates in virtual fire scenarios.For instance, materials like wood with substantial mass burn slowly and necessitate a longer duration of virtual fire extinguishing with a larger amount of powder.Conversely, materials such as fabric burn rapidly, characterized by higher flame size growth rates and burn rates.

Evacuation Drills in VR
VR evacuation training is designed to create realistic building layouts, and various scenarios that participants may encounter during actual evacuation.This method enables trainees to learn how to act and make decisions with their whole body in a safe and controlled environment and effectively prepare for real emergency situations.This training starts after the completion of the AR training.A portal connected to the virtual environment is created, through which participants can transition to the virtual space [see Figure 1(e)].Although AR enables movement in real space, in VR, collisions may occur when moving.
Therefore, a different movement method from AR was applied in VR.We designed the system to track leg movements through trackers attached to both ankles, allowing forward movement in the virtual space by walking in place (see Figure 6).After transitioning to the VR space, the system is designed to move in place to prevent issues related to external collisions.In this training, the goal of the trainee was to escape according to fire evacuation guidelines.To intuitively convey these guidelines to the trainee, we used a virtual character.This character demonstrates the evacuation guidelines, and trainees should follow the actions of the character to safely evacuate [see Figure 1(f)].During a fire, trainees should lower their head, posture, and cover their nose and mouth with a cloth to minimize the intake of toxic gases concentrated in the upper part of the corridor.Additionally, moving swiftly along one wall is the recommended evacuation guideline.

IMPLEMENTATION AND EVALUATION
In this study, a desktop computer with VIVE Pro HMD and a prototype extinguisher were used to conduct fire drill training.Although the HMD used was originally designed for VR, its built-in stereo camera enables AR functionality.To enable this feature, we used the SRWorks SDK (0.9.7.1) provided by VIVE Developers.The desktop used in the experiment was installed with Windows 10 Pro (64-bit), an AMD Ryzen7 5800X CPU, GTX 1080Ti GPU, and 32 GB RAM.The prototype extinguisher was designed using a VIVE Tracker (3.0), Arduino (UNO Wi-Fi R2 Board), gyroscope sensor (MPU-9265), and switch sensor installed on a 4L water extinguisher for education use (see Figure 5).Finally, we configured the software environment using the Unity 3D engine (2021.4.05.f1).With the  Unity 3D engine, VIVE Pro HMD, and prototype extinguisher, we created a realistic and immersive fire drill training simulation.The VIVE Tracker, Arduino board, and sensors were used to detect the user's movements and interactions with the prototype extinguisher.The system was designed to provide a realistic training experience for users to practice extinguishing virtual fires in a safe and controlled environment.

AI Models Evaluation
The  1 presents the accuracy obtained after testing the MobileNet-v2 model on the data and test sets, and the accuracy obtained after testing on the HMD.When testing on the actual HMD, the accuracy was checked after recognizing five different types of objects from various angles 100 times each.The test set yielded an accuracy of 96.27%, whereas the actual scenario demonstrated an accuracy of 82.4%.This discrepancy is attributed to the lower quality of the captured images obtained from the built-in camera of the HMD compared to the test set.In the case of the recognition time of this model, it was 140 ms, and it only needs to be run once for each object when the object is first recognized.
Table 2 shows the average relative error and accuracy of our mass estimation model.This model measured the error rate by measuring ten times for each of the five types of objects.the average estimation time of this model is 2.4 ms, and like MobileNet-v2, it only needs to be run once.

USER STUDIES
We aimed to gain insights into the utility and future potential of the integrated AR/VR system through usability evaluations in terms of intuitiveness, effectiveness, and immersion, as well as simulator sickness evaluations.Before the experiment, participants were introduced to the purpose and usage of the system for 10 min and provided a preimmersion baseline by completing the simulator sickness questionnaire (SSQ). 17The SSQ is composed of a total of 16 questions, with each question scored on a scale of 0 to 3. The experiment scenario involves training in AR, then transitioning to VR via a portal for evacuation training.The experiment lasted for 5 min, during which time participants were asked to simulate AR/VR firefighting training using the system.Upon completion of the experiment, participants first responded to the SSQ questionnaire, and then filled out the System Usability Scale (SUS) 18 and a self-made immersion questionnaire.The time for the survey was limited to 10 min, so the evaluation was conducted over a total of 25 min.Afterward, a brief interview was conducted on the training system to collect participant feedback.A total of 21 participants with an average age of 31.8 years, comprising 17 men and 4 women, participated in the usability evaluation.Nine participants had no prior experience with VR, whereas five participants had previous experience using an extinguisher.Overall, the user testing phase provided valuable insights into the usefulness, efficiency, and immersive quality of the proposed AR/VR-integrated fire drill training system.Feedback from participants was used to improve the design and functionality of the system.

SSQ Result
To compare simulator sickness for the AR/VR training system, we calculated three subscales for nausea, oculomotor issues, and disorientation, and the total score.Each of the three subscales is calculated as the sum of 7 items, with scores ranging from a minimum of 0 to a maximum of 21.The results showed an overall increase in simulation sickness when comparing the baseline scores responded before the experiment with the scores after the experiment.However, the degree of increase was not severe enough to cause critical discomfort during actual use (see Figure 7).As the test time for this experiment was not long, there is some limitation in deriving clear results.Therefore, it is necessary to measure changes in the experience of motion sickness over extended periods of experience.After the test, three participants reported that they felt a little stuffy and dizzy, saying that the reality looked blurry due to the low image quality of the camera in the test.

SUS and Immersion Evaluation Results
The SUS is a quick and efficient way to test the suitability of a user interface.This method has been proven to be "valid and reliable" as a tool that measures only the "subjective perspective." 19The questionnaire consists of five positive items and five negative items, and it is designed to gauge the level of usability according to the score (see Figure 8).The results of the evaluation were M=88.75, SD=7.03, with a minimum score of 72.5 and a maximum score of 97.5.Through this evaluation, it was found that the system had high utility at an "Acceptable" level.
The immersion questionnaire consists of ten subjective questions, with scores ranging from 1 to 5 (see Table 3).The evaluation showed a positive result with M=83.81,SD=10.17.Through this evaluation, it was confirmed that the system provided a new and interesting experience and satisfying training.In particular, most participants reported that connecting the extinguisher to the virtual environment enhanced immersion in education.They also expressed surprise and interest in the potential of the technology that combines the real and virtual world.

DISCUSSION
Strategies for AR/VR Integration AR and VR experiences exhibit unique strengths and weaknesses.AR can combine the real and virtual worlds, providing a high level of realism and excellent accessibility without isolation from the external environment.However, AR usability can vary depending on the screen size and resolution of the device and may have lower immersion than VR.By contrast, VR exhibits excellent immersion and the ability to represent infinite virtual worlds, rendering it applicable in various fields.However, VR is completely separated from the real world, which severely limits the range of activities in the physical space and can cause motion sickness.We addressed the limitations of each technology and created novel benefits through integration.Through usability testing, we confirmed that the integrated AR/VR technology was useful and interesting to people.However, we found that a low-performance pass-through camera could give a negative experience to the AR experience.Especially to create a natural AR experience in a fully enclosed HMD, a camera and display of very high quality and low latency needs to be required.

Strategies for Real Space-Based Virtual Space Creation
A critical aspect of AR is its ability to interact with the real world.In fire drill training, understanding how fires start and behave in various environments is critical.Fires can start differently depending on factors, such as the type and location of furniture and objects, humidity, and wind strength.For effective training, creating realistic virtual fire situations that simulate these conditions is essential.In this study, a machine learning model was used to recognize detailed properties such as object type, location, volume, and mass, which enables the creation of virtual fire environments that accurately

FIGURE 1 .
FIGURE 1. Scenario description for AR/VR fire drill training.(a) Object recognition in real space.(b) Recognition of detailed features, size, positions, mass, and materials of objects.(c) Generation of a virtual fire effect that matches the detailed characteristics of the object.(d) Extinguishing a virtual fire using a physical fire extinguisher.(e) Moving from real space to virtual space.(f) Learning fire escape tips by following the character in front of the user.(g) Emergency situation in virtual evacuation.(h) End of scenario-safe escape.

FIGURE 2 .
FIGURE 2. Architectural design of our fire training system using integrated AR/VR technologies.

FIGURE 3 .
FIGURE 3. Transiting from the real space to the virtual world through a portal.

FIGURE 4 .
FIGURE 4. Identification of detailed features of objects using various AI models and augmentation of virtual fires.

FIGURE 5 .
FIGURE 5. (a) Full synchronization of physical and virtual fire extinguishers.(b) Virtual fire extinguishing using tangible fire extinguisher.

FIGURE 6 .
FIGURE 6. Natural VR locomotion method through walking in place.

FIGURE 7 .
FIGURE 7.This graph represents the Simulator Sickness Questionnaire (SSQ) scores from the baseline experiment.The left bars in gray depict symptom levels prior to the experiment, while the right bars marked with blue stripes illustrate symptom levels following the experiment.The different symptom categories on the x-axis include: Nausea (N), Oculomotor (O), Disorientation (D), and the Total Score (TS).

FIGURE 8 .
FIGURE 8. Interpretation of individual item results and scores for the System Usability Scale (SUS) questionnaire.

TABLE 2 .
Mass estimation evaluation result.

TABLE 3 .
Immersion evaluation questionnaire (n represents negative responses; p represents positive responses).Strategies for Tangible Props in TrainingUsing real and tangible props in virtual training contributes considerably to the effectiveness of the training experience.We developed a fire extinguisher prototype perfectly integrated with the virtual environment to enhance the effectiveness of the training.Participants in the user test responded that the prototype was highly effective in learning how to use a real fire extinguisher.Although most participants had never used an extinguisher before, they reported gaining confidence in their ability to use an extinguisher in future fire situations because of this training.Integrating real training props into virtual training is a challenging task.Commercial tracker devices are a simple and effective method of tracking the position and orientation of real training props.To obtain additional inputs or external information such as humidity and temperature, we presented an improved method using Arduino and various sensors.Arduino used in this study supported both wired and wireless modes, though the wireless mode had limitations in real-time location tracking.To overcome this problem, either the network design should be improved by limiting the number of data transmissions and frequency per hour or excluding functions that require the transmission of large amounts of information, such as continuous location tracking.CONCLUSIONWe developed a system for integrating AR and VR for fire drill training using a single HMD.A novel method was proposed to create virtual training spaces based on real environments using AI technology; real training props were synchronized with virtual training props to enhance training effectiveness.According to the research findings, integrated AR/VR fire drill training exhibited positive results with high average scores and low standard deviations in both the SUS questionnaire and immersion evaluation.Although no severe simulation sickness was observed in this AR/VR experiment, our findings indicate that low camera resolution could potentially hinder the AR experience.Despite this, the majority of participants expressed interest in the system's potential upon completing the training.We also confirmed the potential to drastically improve training and education in various fields by identifying the advantageous elements of AR and VR and combining these technologies.Additionally, training effects were enhanced through strategies that develop realistic virtual environments using AI technology.However, improving the integration of AR and VR technologies requires a high level of hardware performance.Exceptionally high resolution and low latency are expected to compensate for the limitations of this integrated technology and enhance immersive experiences.In the future, AI agent characters that can interact with users in AR/VR integrated environments can be developed to provide effective and engaging education without requiring experts.