ReImagine Lab: Bridging the Gap Between Hands-On, Virtual and Remote Control Engineering Laboratories Using Digital Twins and Extended Reality

Practical work is one of the most important instructional tools in control engineering. To address concerns linked to the cost and space requirements of traditional hands-on laboratories, technology-enabled laboratory modes, such as virtual, remote, and take-home laboratory modes are proposed. Each of these alliterative laboratory modes has its own set of benefits and emphasizes a distinct learning goal. Furthermore, due to lockdown and physical proximity restrictions imposed by policies in response to the COVID-19 pandemic, the employment of these laboratory modes has been quickly increasing. The laboratories’ development, operation, and maintenance become more fragmented as a result of these many possibilities. In this study, we propose “ ReImagine Lab” as a framework for leveraging digital twins and extended reality technologies to streamline the development and operation of hands-on, virtual, and remote laboratories. By increasing the level of interaction, immersion, and collaboration in technology-enabled laboratory forms, this framework intends to boost student engagement. The benefits of this framework are demonstrated by examining several use cases, and a 37-person “system usability study” is conducted to assess the usability of virtual laboratories employing desktop computers and immersive virtual reality.

persistently highlight the importance of practical hands-on experience in successful control engineering education from the early stages of learning [3].
There are many pedagogical tools that can be applied to build the practical hands-on expertise required by control courses, including course project assignments, internships and laboratory experiments. Practical work in the laboratory (''control laboratory'') has become a standard component of automatic control courses, as they aim to [4]: • connect theory to implementations and observations in the laboratory, • identify differences between models and physical systems, • design and verify controllers that meet specifications, • collect and visualize data. Traditionally, these laboratories were based on working with laboratory-scale control objects that were used to demonstrate dynamic phenomena that was observed in fullscale, industrial counterparts of these objects.
These experimental setups incorporated computer interfaces to enable students to design and tune controllers and observe how the system behaves under these new conditions. For example, in [5], the laboratories included experiments with a coupled tank system, an inverted pendulum and a rotary table. These systems were used to demonstrate the use of modeling, simulation and control design to students.
However, actual physical laboratories are costly to build and maintain. They require a considerable amount of space and are composed of specialized hardware, which increases the complexity of the necessary infrastructure. Moreover, as the number of students increases, managing the infrastructure and organizing physical laboratories becomes extremely challenging. In other words, this infrastructure does not scale well.
Thus, leveraging the recent advances in information and communication technologies, educators have created different alternative modes of technology-enabled laboratories to tackle the issues related to traditional physical hands-on laboratories. Works [2] categorized these different laboratory modes based on the nature of the experimental resources (real or simulated) and the location of these resources (local or remote), as shown in Table 1. Further discussion is largely based on this taxonomy.

A. SEEKING THE ULTIMATE LABORATORY MODE
There is an ongoing debate on the effectiveness of the different laboratory modes. A comparative analysis of the different laboratory modes has shown that when these laboratory modes are developed, their efficiency is measured by their ability to achieve different learning objectives [29]. For example, remote laboratories are more suited for understanding concepts, while virtual laboratories are better suited for learning design skills. This makes it difficult to prioritize any single laboratory mode.
Another important factor to consider is the way that students' interaction with laboratory objects, instructors and other students is affected by the specific laboratory mode. Analysis of results from studies of students' interaction in face-to-face and remote hands-on laboratories have shown that there is a lack of systemic analysis of students' interactions in the alternative technology laboratory modes. Before we are able to understand the full implications of the use of such laboratory modes, we need to have better tools to investigate the students' interactions [30].
While hands-on physical laboratories have evident disadvantages related to cost and space requirements, remote and virtual laboratories also have drawbacks: • In remote laboratories, students report a lack of personal engagement because of the separation between them and the experimental objects [31], [32].
• Virtual laboratories have even more of a separation, as the virtual system does not physically exist and the relationship is not clear between the physical and the virtual environment.
• The usability of virtual laboratories is questioned, as it is not the focus point when designing virtual laboratories [33]. In addition to the learning objectives of control engineering courses, to meet the needs of the industry as well as the accreditation criteria imposed on the university study programs, engineering students should develop not only professional skills but also soft skills. In this case, the working patterns that happen in hands-on laboratories are more suited to foster these skills compared to remote and virtual laboratories [34]. An overview of laboratories in control engineering and their alternatives is presented in Table 2.
Regardless of the ongoing debate, the recent coronavirus pandemic has forced the use of these alternative laboratory modes as physical distancing and lockdowns prevented the use of traditional physical laboratories [35], [36]. The lockdowns and related movement restrictions resulted in the need for organizing classes in hybrid form, and thus, the option of distance learning was made available to students. In a similar way, a hybrid approach combining several laboratory modes has emerged that allowed for the modes to complement each other. For example, virtual and remote laboratories concerned with the same control object can be used as follows [26]: • the virtual mode is applied during the control design stage when no interaction with the real system is strictly necessary; • the remote mode is applied for observing the behaviors introduced by the real system and deepening the understanding of related concepts. As a result, rather than searching for the ultimate laboratory mode, one might devise a method for combining all of the modes within the same laboratory activity.

B. EMERGING INDUSTRY 4.0 TECHNOLOGIES: DIGITAL TWINS AND EXTENDED REALITY
Hands-on laboratories in control engineering are often extended with mathematical models and simulations that, on the one hand, provide the theoretical foundations for VOLUME 10, 2022 modeling the specific control object and, on the other hand, allow the students to design control systems based on these models. The models themselves are often based on approximations leading to unmodeled dynamics. For example, the parameters of the studied systems are assumed to be timeinvariant, yet in real life, the parameters of the systems are subject to change. Thus, updating these models and simulations requires manual effort and specialized expertise, which makes creating, operating, and maintaining the laboratories even more costly.
The industry 4.0 revolution, however, emphasized the usage of data as the cornerstone for improving processes and operations across all industries. For example, industry 4.0 builds on a data-driven architecture by utilizing models that are capable of using data from real systems, which are used to synchronize the virtual representation of these systems with their real life counterparts, leading to the concept of Digital Twins (DT). The most cited definition for a digital twin reads as follows [37]. Note that the quote is taken from the document published by NASA, hence the occurrence of the ''flying twin'' concept, but the applications of the digital twins are obviously not limited to the aerospace industry.
A digital twin is an integrated multiphysics, multiscale, probabilistic simulation of an as-built vehicle or system that uses the best available physical models, sensor updates, and fleet history to mirror the life of its corresponding flying twin.
The digital twin concept has many other definitions in the literature, as it is an emerging technology, and researchers are experimenting with its applications across different types of industries.
In our work, we follow the definition presented by the Digital Twin Consortium: ''A digital twin is a virtual representation of real world entities and processes, synchronized at a specified frequency and fidelity'' [38]. Toward the dynamical modeling aspect of a digital twin, we turn to [39], which describes the digital twin as having three main parts: • a model of the object, • an evolving set of data relating to the object, • a tool for dynamically updating or adjusting the model in accordance with the data.
Concerning the last item, we note that the need for transferring data from virtual simulation to DT-based virtual laboratories specifically has been stressed in [9].

C. CONTRIBUTION
In this paper, we build upon this idea by showing that DT-based laboratories replace traditional simulated virtual laboratories and remote laboratories. Furthermore, with this approach, the traditional hands-on experiments are enhanced by enabling shared and mixed experiences coupled with the use of the extended reality technology. The proposed framework is composed of multiple levels of fidelity based on digital twin implementation and extended reality (XR), with the ability to create unified and compliant modes for hands-on, virtual and remote laboratory experiments: • First, remote laboratories are replicated as digital twins of the original laboratories synchronized at high frequency.
• Second, locally hosted virtual laboratories are digital twins of the original laboratories synchronized periodically to ensure the validity of the virtual representation.
• Finally, XR is used to enable a higher level of interaction and visualization offered by digital twin representation.
A comparison of the characteristics of different laboratory modes and hybrid ReImagine laboratories is presented in Table 3.
The main contribution of this study is presented in the following paragraphs. We first introduce the ''Reimagine Lab'' framework and show how the use of digital twins and extended realities streamlines the creation of virtual and remote laboratory modes. We then review the benefits of using the framework for both remote and virtual laboratories, 89926 VOLUME 10, 2022  as well as how it enables the creation of shared experiments with hands-on laboratories. A specific use case is studied to test the validity of the framework, where a digital twin of an actual control object, a laboratory-scale model of a 3D crane located in the CS laboratory at Tallinn University of Technology, Tallinn, Estonia [40] is created.
The structure of the manuscript is as follows. The proposed DT and XR laboratory framework is detailed in Section II. A use case of a digital twin-based implementation of a 3D crane is described in Section III. Next, the system usability study comparing the use of immersive virtual reality and desktop virtual experiments is put into context and presented in Section IV. The results of the experiments are presented and discussed in Section V. Finally, conclusions are drawn in Section VI.

II. REIMAGINE LAB: A DIGITAL TWINS AND EXTENDED REALITY FRAMEWORK
The proposed framework is underpinned by two major components: digital twins and extended reality. Fig. 1 presents an overview of the proposed framework. The components highlighted by dashed outlines represent the different lab modes (with the exception of hands-on labs): • Remote laboratory mode; • Virtual laboratory mode; • Cloud hosted virtual laboratory mode. In all cases, while multiple fidelity levels of digital twin implementations are used to represent the controlled physical object, XR is used to enable a higher level of visualization and interaction required by the digital twin representation. The components on the bottom left represent: • the physical lab asset, which allows for the hands-on lab mode to be employed; • the big data server that handles data storage for digital twin synchronization and other tasks; • a cloud-hosted version of the digital twin of the physical lab asset. The suggested framework addresses all of the issues raised previously. Table 4 presents an overview of the characteristics of the framework, and each lab mode is discussed separately in the next section.
A. REIMAGINE-LAB MODES 1) REMOTE MODE As illustrated in Fig. 2, the framework performs remote teleportation of the laboratory asset by substituting video streaming with synchronization of the local digital twin of the real asset. Extended reality is being utilized to offer a more intuitive and natural form of engagement using hand gestures and other tools, allowing for an experience comparable to that found in the hands-on laboratories. Because this type of engagement does not need any user to have control privileges, the usage of XR facilitates collaboration by establishing virtual environments where users may interact with one another and the laboratory object. • first, the bidirectional evolving set of data guarantees that updates from the actual laboratory object are applied automatically to the digital twin;

2) VIRTUAL MODE
• second, adhering to the digital twin principle, ReImaginedata that describes the uncertainty and divergence between the digital twin and the physical twin is also available; • finally, usingf XR technology as a medium of interaction to take advantage of the rich amount of information is available through the digital twin architecture. The first two features are intended to foster greater trust in the virtual simulation, while the use of XR enables the creation of environments that promote student collaboration.
Cloud-hosted virtual environments provide additional benefits from the framework because they enable the use of higher-fidelity twin models. As illustrated in Fig. 3, the simulation is distributed across the network, with the local device rendering the visual representation of the digital twin asset while computation is offloaded to the cloud.

3) HANDS-ON MODE
The benefits of utilizing the framework are not limited to technology-enabled laboratories; they also benefit hands-on laboratories by allowing for mixed experiences in various laboratory modes. The use of XR and digital twins in Fig. 4 enables a mixed experience where a group of students can interact directly with the laboratory asset while others can interact remotely. This interaction can be bidirectional if local students are also utilizing augmented reality to interact with laboratory assets.

B. CREATION OF THE REIMAGINE LAB ASSETS
The following section defines the core process of creating digital twins in the context of automatic control systems. This involves mathematical modeling, the creation of 3D assets, interaction and visualization design.

1) MATHEMATICAL MODELING
We first address the modeling and simulation aspect of digital twins. Here, modeling broadly refers to the problem of the coherent representation of the dynamics of the system studied by computing the evolution of its internal variables (states) under external stimuli (inputs). States and inputs represent some physical properties of the system. In general, a dynamic model can be represented in state space form using a system of differential equations as follows: where x ∈ R n is the state vector, u ∈ R m is the input vector, y ∈ R p is the output vector, and t is the time argument, and f (·) and h(·) are nonlinear functions. For convenience, linear, time-invariant approximations of (1) are often used and are of the form as follows:ẋ where A ∈ R n×n , B ∈ R n×m , C ∈ R p×n , and D ∈ R p×m are state, input, output, and direct transmission matrices with numerical entries, respectively [41]. For single-input, single-output linear, time-invariance systems, the concept of the transfer function can be employed. The corresponding dynamics equation in the Laplace domain is given as follows: where s is the Laplace operator, and a i and b j are real numbers, and n is the order of the model. For the system in (3) to be practically realizable, it must be proper, i.e., the condition n m must be satisfied. In terms of modeling approaches, the usual ''box'' models are considered: • White box modeling (also known as First Principles modeling). The structure of the model is known, and the model is derived from physical laws.
• Gray box modeling. The model is partially derived from physical laws. Certain parts of the model are approximated such that these approximations have no direct physical interpretation but are nevertheless suitable for modeling purposes.
• Black box modeling. No information about the physical structure of the system is given a priori. As a result, the model is created by fitting experimental data to a mathematical type of model and structure that has been arbitrarily chosen. This is a data-driven technique that is ubiquitous, although it may be less beneficial if the structure of the systems under investigation is known.
In the case of gray and black box modeling, data must be collected from real life plants. In the present work, data are collected by sampling the sensors of the real life plant.
The system under investigation is connected to a desktop computer through a data acquisition device that allows the collection of all relevant data for creating a mathematical model of the digital twin. The complete process is presented in Fig. 5.
After the collection and preprocessing of data, the model identification procedure is carried out. In this work, we consider linear approximations of the system in question and for the linear models in (2) and (3), the estimated parameter sets are as follows: and respectively, with the individual entries of θ ss and θ tf representing row vectors of parameters stemming from either the VOLUME 10, 2022 corresponding system matrices or the zero and pole polynomials. Time identification is employed such that the output error criterion (residual norm) is as follows: is minimized, where ε i = y i −ŷ i is the residual (simulation error), y i is the true system output andŷ i is the predicted output for collected samples i = 1, 2, . . . , N .
In the case of a multi-input, multioutput system, the residuals resulting from modeling individual outputs are scaled according to the magnitude of the modeled physical variable, and a weighted sum is used as the cost function. Several optimization algorithms are used to estimate the parameters of the model, including the Trust Region Reflective algorithm [42], [43], the Levenberg-Marquardt algorithm [44], [45], and the Nelder-Mead direct search method [46]. The latter is well-suited to optimize a function with derivatives that are unknown or nonexistent. Addressing the problem of the initial parameter estimation, the subspace estimation method is used [47].
Concerning control, in the present work, we consider the classical negative unity feedback control loop as follows: consisting of a controller denoted by C(s) and the plant denoted by G(s). Here, the objective of the control system is to manipulate the plant input u via the controller to minimize the error e, i.e., difference between the desired output r (reference value) and the true output of the plant y, and we consider the output tracking problem. In real life industrial applications, a proportional-integral derivative (PID) controller is typically used [48], [49], [50]. In this work, we employ the parallel form of the PID controller that has the form as follows: where K p , K i , and K d are the proportional, integral, and derivative gains, respectively. These parameters must be properly tuned for each control loop that composes the full system. An important point to make is that in the case of digital twin synchronization with real systems, the parameters of the models obtained using the methods described above are not static and coevolve with the changes in real systems. Therefore, the process depicted in Fig. 5 can be thought of as the creation of a mathematical model snapshot. At preset time intervals, new model snapshots are created. Hence, the process of parameter estimation and controller parameter transfer is continuous. As the specific physical lab asset is utilized, it generates valuable data that are stored on the server and used to obtain the updated mathematical models.

2) 3D MODELING
All digital twins of control objects are recreated by either measuring the dimensions of the parts of physical devices or by using available blueprints and then implementing the 3D models using CAD software, such as the Blender 3D modeling software [51]. For XR applications, efficient realtime rendering of the objects must be ensured. Therefore, the following important considerations are in effect when modeling all objects: • All 3D models must be optimized, i.e., the number of polygons forming the part reduced and visualization tradeoffs sought in terms of applying textures, displacement maps and lightmaps.
• A sufficient level of detail must be ensured such that the effect of immersion is achieved [52]. The complete procedure for 3D modeling thus is composed of the following: steps: 1) Measuring the physical devices or using a previously known blueprint data; 2) 3D modeling in Blender ensuring a sufficient level of detail is achieved; 3) Optimization, meaning the application of necessary textures, baking displacement-and lightmaps; 4) Exporting the 3D model from Blender into a common 3D asset exchange format (usually FBX); 5) Importing the 3D asset into the real-time rendering engine, creation of materials that are used on the 3D model, validation in the target extended reality application. If necessary, we may return to step 2 to correct any issues discovered in the real-time application.
The process of 3D modeling can also be semiautomated by introducing photogrammetry [53]. This approach, however, falls outside the scope of the present paper.

3) PROTOTYPING PLATFORM
To efficiently codevelop the 3D visualization and XR and the mathematical modeling parts, a coherent prototyping platform is needed. In the present work, the following software packages are chosen to implement the platform: • Unreal Engine 4 [54] as the visualization platform due to highly sophisticated support for virtual reality and rapid game logic prototyping via Unreal Engine Blueprints.
• MATLAB/Simulink environment [55] as the mathematical prototyping platform with real-time simulation support via the Simulink Desktop Real-Time toolbox.
• UDP communication plugin for Unreal Engine 4 that makes real-time simulation possible and was developed for Re:creation Virtual and Augmented Reality Laboratory-related applications [56].
The diagram showing the prototyping configuration is depicted in Fig. 6. This configuration allows for true realtime simulations to be carried out. Prototyping involves the following stages: 1) Development of mathematical models based on the methods discussed in Subsection III-B. Design of mathematical models of interaction mechanics. Validation of the models using data from real control objects. This part is done in either the MATLAB or Simulink environment. As a final stage, functions or blocks enabling real-time data communication through the UDP protocol are added to the project. 2) Development of the 3D models per the methods discussed in Subsection II-B2. After importing the models into Unreal Engine 4, correct assembly of all parts in the hierarchical structures follows. This has to do with ensuring correct coordinate transformations to be applied to connected parts of the given object.

3) Evaluation of the developed application in virtual real-
ity. Assessment of the immersion effect, correctness of dynamics and interaction mechanics.
Once refined, the mathematical models can be directly exported from Simulink as C++ code and integrated into Unreal Engine 4 as blueprint-accessible code plugins. This approach provides the greatest amount of flexibility because the developed mathematical models of dynamics are computed in separate modules that are accessible as blueprint blocks with the required number of inputs and outputs. The plugins are also reusable in other projects.
The prototyping platform can also be used to teach control system design effectively. In this case, the student receives the Simulink block, which represents the system and internally implements communication between the mathematical model and visualization. The visualization application can then be kept running at all times while the mathematical model with the designed controllers is launched several times to enable experimentation with different controllers or controller settings. This can also be done as part of group work, with one student controlling the experiment from the VR environment and the other designing the control experiment in MATLAB/Simulink.

4) INTERACTION DESIGN
Interaction is the most important aspect of an immersive XR environment. While developing digital twins of control systems, the design of meaningful interactions is the main goal of the training aspect of the application [57]. As a result, the development of coherent interactions is regarded as a top priority for ensuring effective laboratory instruction.
In this work, we explore two types of interactions that arise in the area of control systems: VOLUME 10, 2022 FIGURE 5. The process of creating a data-driven mathematical model for the digital twin. Relevant data are first collected from the real plant. Then one of the box models is used with system identification (the choice is determined by the position of the switch in the figure). Finally, the digital twin can use the model of the dynamics. The model is periodically updated in a process referred to as synchronization of the real system and the digital twin. 1) Interactive selection of the control system tracking reference (set point); 2) Interactions with floating information panels that display valuable data concerning the setup and the state of the laboratory experiment. Next, we focus on key aspects of the implementation of these interactions. There are several options available when designing interactions. First, we can implement those using the physics engine available in the target platform. In this case, the mathematical description of the process is largely unclear. The task is, however, to obtain a valid mathematical model of the whole system, including interactions, which must be reproduced in the digital twin. Thus, interaction design is also seen as a mathematical problem and all modeling methods discussed in Subsection III-B are valid for this purpose. Several methods are used for developing interaction mechanics: • Interactions are coupled with the object dynamics, that is, the corresponding (non)linear mathematical model is augmented with corresponding inputs and states; • Interactions are decoupled from the object dynamics, that is, a separate mathematical model is designed for the interaction. This approach is feasible only if the interaction does not affect the control system performance, and thus, its use is usually limited.
• An interaction is designed for the supporting components of the XR experience (such as using the information panels). Mathematical models of these interactions are, at first glance, not needed; however, if one considers the concept of intelligent immersive virtual environments (IIVEs), useful intelligent mechanics can be employed as well [58]. Interaction mechanics are first evaluated by comparing the performance of the model with that of the original control object. Then, the subject-based evaluation is performed in XR internally by developers and through subject-based experiments. If the results are not satisfactory based on the feedback, the mechanism is revised.
Another important interaction mechanic is not considered in the case study presented in this work, but it should be mentioned. This is the direct physical interaction with the moving parts of the recreated control objects. From the control systems perspective, this is generally used to introduce disturbances into the studied systems. From the user perspective, such interactions are of curiosity driven experimental nature, and hence, are very valuable features.

5) GRAPHICAL DATA ANALYSIS
Graphical representation of data is a very convenient tool for analyzing the underlying phenomena [59]. Consequently, one of the key aspects of learning control system dynamics is related to the study of time series charts depicting system dynamics [41]. For this reason, the corresponding feature must be implemented in the XR visualization, that is, a realtime time series chart must be available. Therefore, the following items are considered: • Due to the flexibility of presenting data in XR, the graphs can be presented to the user upon request and attached to the view port in an unobtrusive way. For example, the dynamic chart may be attached to one or both of the motion controllers and shown upon the user pressing a preset button; FIGURE 7. Typical time series chart for studying control system dynamics, here for a single input single output system. Top: tracking performance analysis through step response evaluation. Bottom: control law dynamics analysis (system input generated by a controller).
• The structure and types of charts shall depend on the particular study. In studying control systems, one is generally interested in control system tracking performance and control law behavior. Thus, the most general chart is presented in Fig. 7.
In this work, for implementing charts in the XR application, an Unreal Engine 4 plugin called Kantan Charts is used [60].

III. CASE STUDY: DIGITAL TWIN BASED IMPLEMENTATION OF A 3D CRANE IN EXTENDED REALITY
Hereinafter, a case study of developing a coherent digital twin of a lab-scale model of an overhead crane is provided in the context of the proposed framework. The original real life control object was produced by the Inteco company [61] and is commonly referred to as the ''3D crane'' as a reference to the number of degrees of freedom involved in moving the payload. The 3D crane is a nonlinear electromechanical system that possesses a complex dynamic behavior and creates challenging control problems [61], [62]. The industrial counterpart of this laboratory model is used in various industries and seaports to transport large and heavy containers and other payloads. To ensure efficiency and productivity, the crane must transport the payload as fast as possible to its destination. However, a certain motion profile must be employed such that the control actions leading to the acceleration and deceleration of the payload ensure secure and sway-free transportation [63]. The characteristics of the system allow the application of various control strategies [62], [64], [65]. This makes it very appealing as an educational tool in the control systems laboratory.
The present control object is depicted in Fig. 8. It consists of a frame, a moving rail attached to a moving cart. The payload is attached to the cart via a rotating spool. Thus, three degrees of freedom are achieved. The rail, cart and payload spool are actuated by DC motors, and their positions are determined with incremental encoder sensors. In addition, the two encoders are attached to the cart that measure the swing angle of the attached payload.

A. 3D MODEL OF THE OVERHEAD CRANE
Following the discussion above, the 3D model of the crane is developed. The following major components are recreated: • Yellow frame; • The moving rail; • The moving cart with the moving spool; • The payload itself attached to its cable. Thus, all critical mechanical components and the frame have been faithfully recreated, while the wires, DC motors, encoders, pulleys and belts were ignored. It was confirmed through initial experiments that as long as the recreated components had the correct scale and behaved exactly as expected, the 3D model would be convincing enough for immersion to occur [58]. In the future, the other components can be recreated as well, but the additional complexity may not necessarily benefit the present digital twin. The resulting model is shown in Fig. 12.

B. MATHEMATICAL MODEL OF THE 3D CRANE
The discussion below pertains to obtaining a single snapshot of the physical twin dynamics using the methods described in Sec. II-B1. The model shall be updated periodically based on the data generated during the operation of the physical overhead crane.
In this work, we use the physical model of the object shown in Fig. 9. Instead of using a complicated nonlinear model as in previous cases, linear models are used for two purposes: • Describing the motion of the rail and the caret in the (x, y)-plane (transfer functions); • Determining the dynamics of the payload swing angles α and β (state space model). The third degree of freedom (payload height) is not used. The payload is fixed at a height of approximately 30 cm from the floor. Time domain identification is used to obtain a snapshot of a decoupled set of models. For the transfer from VOLUME 10, 2022 the normalized actuator control signal u x , u y ∈ [0, 1] to the rail and caret positions, we obtain the following: and and for the transfer from the inputs to the swing angles the state space model of the form (2) is obtained, where the state matrix is as follows: The input matrix is as follows: The output matrix is as follows: and the direct transmission matrix is as follows: Furthermore, the model in (11)- (14) was modified so that the second swing angle motion needed to be corrected. This was done by multiplying the real part of the corresponding eigenvalues by a factor of 2.5. Then, a balanced reduction technique was applied to the modified model, which also resulted in a nonzero matrix D. The corresponding validation plot is shown in Fig. 10. Some modeling discrepancies can be observed. However, when the digital twin of the 3D crane is observed in the XR environment, the modeling errors do not, generally, result in breaking the effect of immersion; the dynamics of the crane are perceived by the subjects as believable. For the present work, the accuracy of the model is not critical as long as immersion is achieved because the goal of the experimental study is not related to demonstrating the precision of the mathematical model but rather demonstrating some highlevel control concepts. However, when further experiments are designed, a different modeling approach should be used. Therefore, instead of using the black box route in Fig. 5, which yields a linear approximation for a fixed line length of the 3D crane, the gray box route should be used instead of involving a nonlinear model of the system. That way, a more precise model can be achieved, the state variable corresponding to the line length can be integrated into the model, and relevant experiments that are, e.g., related to controller tuning, can be designed and carried out in the real environment and with the digital twin.
Finally, although the identification procedure for the models in (9)-(10) and (11)-(14) is carried out separately, since the inputs are the same in both cases, the mathematical model can be combined into a single 8th order state-space formulation for convenience.

C. 3D CRANE INTERACTION DESIGN
The following two interaction mechanics have been implemented for the experiment with the 3D crane: • Interaction with the control object variables: changing the set-point, which refers to the desired location of the crane's payload-and changing the control mode of the crane; • Interaction with the plot widgets: moving them to the predefined locations, grabbing and moving them to a new location, or grabbing and throwing them anywhere in the virtual environment.

IV. EXPERIMENTAL VERIFICATION OF A DIGITAL TWIN OF AN OVERHEAD CRANE MODEL IN EXTENDED REALITY
The goal of this section is to show evidence of a successful implementation of an overhead crane digital twin that can be experienced in an extended reality environment. We cover a single lab mode in the experiment, namely, the virtual lab mode; however, because the digital twin and extended reality technologies underpin all of the other lab modes as well, the data obtained through subject-based testing should contain information about the usability of these technologies in general for the intended application. As a result, if the data show that the digital twins and extended reality are effective in the intended application, then this result can be valid for all other lab modes.

A. THE DESCRIPTION OF THE LAB EXPERIMENT
The experimental configuration is shown in Fig. 11. The main control loop addresses the position of the payload in the (x, y)-plane. The task is to transport the payload from one point to another as fast as possible. The secondary loop compensates for the payload swing and can be turned on and off; the goal of the experiment is to assess the performance of the control loop in both these cases. A screenshot from the application is depicted in Fig. 12. Here, the user points the motion controller away from the reference cube, so the set point is unchanged but is shown on the floor in the form of a crosshair. The charting facility in this case serves as a reference for the performance of the control loop with and without swing compensation enabled. An exemple depicting the situation when the swing compensation is enabled is shown in Fig. 13. By introducing control actions that lead to some oscillations in the caret position, the swing is effectively damped. The specific parameters of the PID controllers are not shown to the subjects in the experiments. Tuning the PID controllers is FIGURE 12. Screenshot from the VR-based 3D Crane application. The user is pointing the motion controller to an area outside of the reference box, so the crosshair appears only to show the current set point. a topic for a different kind of experiment along the lines of what was presented previously in [66].

B. VALIDATION OF THE SOLUTION WITH A SYSTEM USABILITY STUDY WITH SUBJECTS
In the educational setting, cognitive ergonomics is the study of the design of learning activities that conform to students' cognitive capabilities by applying principles based on human perception, mental processing, and memory to improve the usability of the learning activities. Since we are interested in understanding the usability of using VR in control systems courses, we conduct a system usability study (SUS) to compare a classical experiment that introduced the concept of automatic control application for the 3D crane object and a similar experiment in VR. The original experiment uses an interactive Simulink environment enhanced with a 3D model of the crane. The VR experiment uses a digital visual twin of the 3D crane in Unreal Engine driven with a mathematical model implemented in the MATLAB environment.
The experiment is divided into three parts. First, a presentation created by the course instructor introduces the participant to the experiment and the 3D crane control object.
The second part is a classical control course experiment conducted on a desktop computer where a Simulink model of the 3D crane and swing compensation PID controller is presented. The Simulink interface shown in Fig. 14 is displayed on one screen, while the second screen shows graphs that display the model's real-time response and a 3D model of the crane that is moving in real-time based on data received from the Simulink model shown in Fig. 15.
The third part is a similar control course experiment conducted in a VR laboratory environment that includes the ''3D Crane'' control object, a laboratory-scale simplified model of a gantry crane produced by Inteco and recreated as a digital twin in VR, as well as two interactive plot widgets; the first widget shows real-time data representing the 3D crane dynamics and the other graph shows an explanation of the control object parameters. Fig. 16 shows the different elements of the VR laboratory.
For the desktop experiment, we used a laptop computer connected to two monitors. The Simulink model was shown on the first screen, and a 3D presentation of the 3D crane that was created using Unreal Engine was shown on the second screen. Table 5 demonstrates the component configuration of the desktop PC.
To create a VR environment, we used an HTC VIVE Pro Eye virtual reality headset. The HTC VIVE Pro Eye HMD features dual-OLED displays with a combined resolution of 2880 × 1600 pixels and precision eye tracking sensors. In addition to the headset, we used two HTC Vive Controllers that track the location of the user's hands and receive input commands. The tracking area was set up with two sensors,  and the size of the tracking area was approximately 3 meters by 2 meters. The headset was connected to a PC that hosted the virtual environment. Table 5 shows the component configuration of the VR PC.
Our study included 37 participants (20 male, 17 female; an average age of 25.0 years old). Table 6 summarizes the distribution of participants according to several key variables.
All test participants were given the same instructions that are described below.
All three phases of the study took place in the same room. The participant was given a summary of the three main activities they would perform, as well as the sort of data that would be collected. They were requested to sign an informed consent form after agreeing to participate, which specifies the three parts of the experiments, as well as the nature and the extent of data usage and their right to quit the tests at any time.
Participants were directed to a desktop computer where the presentation was shown once they were ready. Participants were encouraged to go over the slides and ask questions if they had any questions.
Once the participant indicated that they had finished going through the slides, they were presented with the second part of the experiment and given the following instructions: 1) Select run in real-time (from the top menu) and flip the first switch (DOUBLE CLICK); 2) Observe the animation. Notice how the load on the 3D crane keeps swinging from side-to-side; 3) After approximately half a minute, flip the first switch back;  4) Now flip the second switch (OFF / ON) and observe the 3D crane. Additionally, observe the time series plots generated on the virtual scope screen and make conclusions about which mode would work best for in a real life scenario. When the second part of the experiment was finished, the participant was guided to an area in the same room where the VR HMD was located.
In the third part of the experiment, a set of steps that served as an introduction to the VR controllers and HMD, as well as performing eye calibration, which allows the capturing of the participant's gaze direction, are performed.
1) An introduction to the VR headset and controllers is given; 2) The headset is put on and adjusted so the display is centered in the view; 3) The controllers are located and picked up; 4) The eye-tracker is calibrated: a) The headset is adjusted vertically so that the display is centered on the eyes; b) The lens distance is adjusted based on the participants' eyes; c) The participants are asked to follow a set of dots using only their eyes. 5) The operator starts the experiment. At the beginning of the experiment, the participants are transferred to a virtual laboratory environment where they can see the ''3D Crane'' control object, a laboratory-scale simplified model of a gantry crane produced by Inteco and recreated as a digital twin in VR, as well as two interactive plot widgets; the first widget shows real-time data representing the 3D crane dynamics and the other graph shows an explanation of the control object parameters. Fig. 16 shows the different elements of the VR laboratory.
During the experiment, the participant's first task was to walk to a predefined location adjacent to the control object. This location was clearly marked in the VE. Once the participants reached the marked location, they were free to do any of the following actions: • Interact with the control object (change the set-point, i.e., the desired location of the crane's payload-and change the control mode of the crane); • Interact with the plot widgets (move them to the predefined locations, grab and move them to a new location, or grab and throw them anywhere in the VE).

C. QUESTIONNAIRE AND SYSTEM USABILITY SCALE (SUS)
After participants finished the third part of the experiment, they were asked to complete a questionnaire of 10 SUS question items for the desktop experiment and 10 SUS question items for the VE experiment with three additional questions about their confidence level in the VR, IT and control systems. The system usability scale includes 10 items with five responses that range from strongly agree to strongly disagree. The example questionnaire includes the following: I found the system was easy to use, and I would imagine that most people would learn to use this system very quickly. To examine perceived task loads: 1) I think that I would like to use this system frequently.
2) I found the system unnecessarily complex.
3) I thought the system was easy to use. 4) I think that I would need the support of a technical person to be able to use this system. 5) I found that the various functions in this system were well-integrated. 6) I thought there was too much inconsistency in this system. 7) I would imagine that most people would learn to use this system very quickly. 8) I found the system very cumbersome to use. 9) I felt very confident using the system. 10) I needed to learn a lot of things before I could get going with this system.

V. RESULTS
The SUS elements are divided into two categories: positive and negative. Even items are negative, while odd items are positive. To acquire the actual score of the SUS results, we deduct 1 for each of the users answers for the five odd components, then subtract the user answer from 5 for the even components. Finally, we multiply all the components by 2.5 to obtain a score in the range of 0 to 100. The quartile distribution of the SUS score for all participants in both experiments is shown in Fig. 17. The average SUS score for all participants in the desktop experiment was 70 with a standard deviation (SD) = 20.8279, while the average SUS score for all participants in the VR experiment was 85 (SD = 10.2977). This shows that the suggested solution's usability is superior to that of the desktop experiment. Furthermore, the lower SD suggests that in the VR experiment, there was more of an agreement on the system's usefulness.
Further analysis was conducted to determine how the selfreported participant distribution affected the usefulness of both experiments. First, as shown in Fig. 18, the system usability scale results, which were categorized based on participants' self-evaluated confidence in using VR, revealed that users who reported being confident in using VR had the highest average usability score. This finding reveals that as users become more comfortable with virtual reality and their confidence grows, the system's usability will improve.   These findings support the use of VR experiments in a broader context throughout the control systems course.
Second, we examine whether individuals who are confident in their overall IT skills are more likely to favor the VR option. The average SUS results for participants with greater levels of confidence in general IT skills and knowledge were higher in the case of the VR solution, as shown in Fig. 19, whereas the average score for the desktop experiment did not change significantly.
Finally, we looked to see if the participants' level of confidence in the targeted study material had an effect on the SUS results in this instance control system. The findings of the system's usability scale were categorized based on the participants' self-evaluated confidence in control systems (Fig. 20). The results show that in both the desktop and VR experiments, participants who reported neutral confidence with control systems provided the greatest usability score. In general, the VR experiment was rated as being easier to use than the desktop trial by the participants. While the limited sample size restricted any conclusions taken from this research, the findings highlighted the need to leverage virtual reality in the creation of more realistic rich experiments for control object digital twins.

VI. CONCLUSION
In this paper, a digital twin and an extended reality-enabled framework for constructing control system laboratory modes is developed and examined as a full framework integrating all lab modes. The developed solution fits naturally into the scope of Industry 4.0 in the context of these emerging digital technologies, each having an important role in transforming the manufacturing landscape. We thoroughly explain how virtual and remote laboratories can be recreated as digital twins of physical control objects. Incorporating extended reality into the proposed digital representation allows for greater interaction with the object while also allowing students and instructors to collaborate with one another.
To verify the main innovation in the proposed contribution, a case study was conducted with a laboratory model of an overhead crane. An immersive virtual reality simulation was created for the crane using the proposed framework. A subject-based experiment was then designed focusing on the usability of the proposed solution versus a traditional desktop-based environment. For this, a typical lab assignment was considered part of the subject-based testing. There were 37 participants involved in the study.
The main conclusion based on the conducted study is as follows. It was successfully confirmed that the proposed framework, from the perspective of combining the digital twins and extended reality technologies, substantially improved the usability of the simulated laboratory environment. While only the virtual lab mode was considered and advanced features, such as remote collaboration, were not addressed, it is possible to say that the technologies supporting the proposed framework have a high potential to improve lab work outcomes for students.
The ability to manipulate the control object and study the outcomes from several perspectives was identified as a critical factor during the design and development of the simulation. This is not particularly surprising because hands-on labs have known similar favorable qualities. However, when implemented in extended reality, new possibilities emerge to provide a more complete experience. For example, the user can manifest and position a time series chart in the surrounding space near the control object. The chart allows us to interpret the results of the experiment from a time domain analysis perspective, which is common in industry. The solidification of this important connection between the time series and actual events can be achieved naturally with the proposed solution.
Future work must be concentrated on implementing the framework in its entirety. Additional subject-based tests with larger sample sizes will also be conducted as the restrictions related to the COVID-19 pandemic are fully lifted.
Furthermore, the design of advanced control system experiments must be done in accordance with the desired learning outcomes of the related study courses. One pressing issue in the industry is the ability to coherently tune PID controllers subject to certain performance specifications. It is expected that the proposed framework will positively influence the ability of the students to manipulate the parameters of the PID controllers in hands-on XR experiments toward achieving better performing control loops with both digital twins of control objects and real objects, as demonstrated in [66].
To conclude the paper, we would like to acknowledge the contribution from Ms. Dolores Freiberg for her work on the 3D crane model described in this article. Additionally, we would like to express our gratitude to Ms. Oleksandra Zamana for assisting us in safely conducting the experiments with the subjects during a particularly trying period of the pandemic.