A Motion Similarity Measurement Method of Two Mobile Devices for Safety Hook Fastening State Recognition

Fall from height (FFH) is an accident that leads to fatalities in construction workers, and a major cause of FFH is due to the improper fastening of a safety hook of a safety harness to a temporary structure. In this study, we propose a new approach for recognizing the fastening state of the safety hook based on the similarity of motion between the motion of the hook and the body. We first assume that the similarity of motion between a hook and a body will be more similar when a hook is fastened to a part of a body than when the hook is fastened to a temporary structure. Under this assumption, we propose a new method that measures the similarity of motion of a hook and a body. In the proposing method, motions are represented through acceleration and rotations of the hook and the body. The magnitude of acceleration is represented as an ordinal variable and the magnitude of acceleration is jointly represented with rotations in a spherical coordinate system for effective similarity measurement of both motions. The effectiveness of our approach is verified by our newly collected task-related human activity dataset comprising the motion data of the hook and the body from inertial measurement unit (IMU) embedded mobile devices. Our proposed method confirmed that representing the magnitude of acceleration as an ordinal variable shows improved performance of 82.95% in terms of Youden’s index. Moreover, it further verified that jointly representing the magnitude of acceleration and the rotation in the spherical coordinate system shows improved performance of 90.64% in terms of Youden’s index.


I. INTRODUCTION
Occupational accidents occur frequently every year, and various efforts to prevent them are being made worldwide. In particular, the accident rate at construction sites account for a large proportion of all industrial accidents across many countries. The proportion of fall from height (FFH) accidents among others is largely due to the specific nature of The associate editor coordinating the review of this manuscript and approving it for publication was Usama Mir .
construction sites as compared to other industrial sites in that high-altitude work comprise a large proportion of the total work, and the majority of such accidents results in fatal injuries. According to statistics on fatal occupational injuries [1], [2], it was confirmed that among all deaths, deaths due to falls comprised 31.1% in the UK [2] and 44.5% in the United States [1], respectively.
FFHs at construction sites are caused by various factors such as environmental factors, task-related factors and personal factors [3], [4]. They can be caused by a single factor or an interaction of the multiple factors, and in the majority of case of FFHs, workers were not wearing their personal protective equipment (PPE) or using it improperly [5]. Although PPE is the last resort for avoiding fatal injuries by a FFH, it is not always used by construction site workers. There are a variety of reasons as to why workers do not use PPE, and a major reason is based on their personal decision [6]. Thus, the worker's personal factors contribute to the frequent occurrence of fatal injuries by a FFH.
Proactive methods are important on-site precautionary measures that are required to be taken before FFH accidents occur, and they are more important as they include on-site precautionary measures [7]. Some authors proactively strategize to avoid such accidents by changing the organization of workers through revised task-planning or improved training regimes for workers [8]. In contrast, some authors have focused on mitigating FFHs using personal protection equipment (PPE). As an example, FFH injuries would less likely to occur for a scaffolder with a safety hook of a safety harness that is fastened to a scaffold. In China, scaffolders are required by law to wear safety harnesses at construction sites, as using PPE minimizes injuries from FFHs, which could otherwise be fatal [6].
In order to prevent fall accidents, it is necessary to ensure that the workers fasten their safety hook to temporary structures when working at heights. On-site safety managers and technicians can supervise workers, but it is difficult for safety managers to monitor workers at all times [9]; thus, a technical agent is necessary [7].
Several prevention strategies have been studied for preventing the occurrence of FFH injuries at construction sites. In some studies, FFHs were prevented by providing improved work routines for workers [6], [7]. In contrast, in some studies, preventative measures by leveraging technological agents have been suggested, such as a radio frequency identificationbased (RFID) [9], [10] and a computer vision-based [11] approach to monitor the presence of safety harnesses and an accelerometer-based approach to monitor the presence of safety helmets or the proper usage of safety helmets [12]. However, these studies only recognize the presence of safety harnesses or the proper usage of other PPEs. Despite safety harnesses being one of the important PPEs for workers working at heights, few studies have been conducted for recognizing the proper use of safety harness. Therefore, in this study, we propose an approach for recognizing whether a safety harness, one of the most important PPE required for workers working at heights, is being used properly. Based on the proposed approach, we attempt to recognize whether the worker is in a state where the safety hook is fastened to the temporary structure or that where the safety hook is fastened to their body.
The concept of this study originates from the fundamental assumption that data from two motion sensors located at different parts of the body will be more similar than a case in which one of the sensors is located at the temporary structure. To extract information regarding the motion of a body and a safety hook during construction site work, an inertial measurement unit (IMU) is attached to each of the safety hook and chest strap of the safety harness.
In contrast to the aforementioned studies that were focused on recognizing the presence of a safety harness, this study is conducted for recognizing the proper use of a safety harness.
This research contributes to the following: 1. The first proposed safety hook fastening state recognition system based on two IMU sensors and the publication of the obtained dataset for future research related to this field.
2. Introduction of a new motion similarity measurement method with a new motion representation method in which the magnitude of acceleration and rotations can be jointly represented on the same coordinate system effective for safety hook fastening state recognition.

II. RELATED WORKS A. EXISTING METHODS FOR SAFETY HOOK FASTENING STATE RECOGNITION
Various designs and approaches for recognizing the fastening state of the safety hook have been proposed utilizing various sensors [13]- [17]. Gómez-de-Gabriel et al. [13] proposed a safety monitoring system using Blutooth low-energy beacons; the relative position of a worker is determined based on the position of the beacons and the receiver, which is attached to a safety harness. When the distance between the receiver and the beacon in a dangerous area is small, it can be deduced that the worker is in a dangerous area. Concurrently, if the distance between the beacon in the karabiner and the receiver is small, it can be observed that the worker is using the safety harness properly.
A similar detection method was proposed in which utilized RFID tags. RFID coupling detectors are attached at the anchor point and the safety harness [15]. However, the aforementioned methods operate on sites where the ambient sensors (RFID tag, Beacon) with anchor point information are available. Moreover, some cases have confirmed that construction site materials interfere with RFID [18].
On the other hand, methods that can be used without anchor point restrictions have also been proposed. In [14], a retrofittable load sensor was attached to the safety hook. The system incorporated its information from the identity (ID) card of the worker, which contained the load information, anchor point information, and anchor point at the construction site. The load information is obtained from the coupling detector attached to the safety harness; and the anchor information is obtained from the coupling detector, possibly an RFID reader, attached at the lanyard of the safety harness. Likewise, in [16], multiple load sensors were used, and all but one were located at the safety hook, which was located at the hook hanger of the safety harness. Each sensor attached to the safety hook was responsible for checking the fastening status, lever status, and disconnection status; the controller then outputs a sound and a light signal based on the status of the load sensors. In contrast to previously mentioned systems, the system [16] informs both the worker and on-site VOLUME 10, 2022 supervisor. However, the systems proposed in [14] and [16] required that the load threshold be set manually, which varied with different workers, to classify the status. Although the threshold of [16] is variable, it requires the user to input the correct information using an ID card. In [19], a single IMU sensor was utilized to recognize the fastening state of the safety hook. They trained various machine learning models based on kinematic parameters from the IMU sensor. In [17], accelerometers were used to recognize the fastening state of the safety hook, and two acceleration sensors were mounted to each body belt and safety hook. When the value of the relative acceleration of the two accelerometers is higher or lower than a predetermined threshold, the state can be recognized by generating an alert signal. Several approaches have been proposed with the use of various sensors. However, most of the above studies are patents, and it is difficult to determine the specific method or analytical experimental results in detail [14]- [17]. Although Gómez-de-Gabriel et al. [13] described their approach with detailed analysis, in which the operating location was limited to near the anchor points where an ambient sensor was attached.
For the monitoring system to be used generally at a construction site, it should be able to be used anywhere there is a temporary structure without restriction on the position of the anchor. We proposed an approach that can be used anywhere there is a temporary structure by recognizing the fastening state of the safety hook using only two IMU sensors attached to the safety harness. Although the study of Muhammad [19] et al. targeted the same application as ours using a single IMU sensor; the approach is different from that of ours. Furthermore, we experimentally verified the effectiveness of using not only the acceleration [17] but also rotation for the safety hook fastening state recognition, and we publish a newly collected task-related human activity dataset for future research related to the prevention of FFHs at construction sites.

B. STUDIES ON THE SIMILARITY OF TIME SERIES DATA ACCORDING TO TWO SENSOR POSITIONS
This study classifies the fastening state of the safety hook based on the similarity measured between the motion of the safety hook and tne body. In this regard, this study is similar to the problem of classifying whether two sensors are located in the same place (person) or in different places (person). Previously, studies to classify whether two mobile devices are located in the same or different places for pairing two mobile devices have been proposed.
Lester et al. [20] attempted to classify whether two devices are located on the same person or on different people. To this end, each device included a three-axis accelerometer, and the similarity in the magnitude of acceleration of the devices was measured to recognize whether they were located on the same person. To measure the magnitude of acceleration of the three-axis acceleration, a one-dimensional magnitude of acceleration was extracted by taking the sum magnitude of vectors (SMVs), and the similarity of the magnitude of accel-eration of the two devices was measured through coherence.
To solve similar problems, Mayrhofer et al. [21] and Rainhard et al. [22] used coherence in a similar manner to that of [20]. Moreover, Mayrhofer et al. confirmed that the area under the receiver operating characteristic (AUC) exhibited the highest coherence among various similarity measurement methods. They utilized the rotation information to de-rotate the orientation of two mobile devices yet they did not consider the similarity of motion of the rotational direction. Bosch et al. [23] measured the similarity of motions by calculating the correlation of linear motion using accelerometers and the rotary motion using compass sensors. The correlation of the linear motion and the rotary motion was calculated independently, and the weights that were determined heuristically were multiplied to each correlation, and combined.
In the studies introduced above, the relative motion with the body is fixed by holding the device in the hand or attaching it somewhere else on the body. Moreover, most of the studies measured the similarity of motions that are somewhat monotonous such as walking [20] or shaking [21], [22] based on the magnitude of acceleration. In contrast, the safety hook in this study was not tightly attached to a part of a body, but it is fastened to a part of the body or a belt of a safety harness. Because it is in a fastening state, it creates swinging motions based on the fastening anchor point. Moreover, the construction site is an extreme environment in which various task-related activities occur, and the motions of the task-related activities are complex [4], [7], [24]. The motion of the task-related activities includes not only walking or shaking motions but also rotary motions of the body. Therefore, measuring the similarity of the rotary motion with the magnitude of acceleration can be effective for the fastening state recognition. The similarity of rotary motion has been considered in [23], but heuristically determined weights were required to combine the similarity of the magnitude of acceleration with the similarity of the rotary motion. It is expected that a training dataset is needed to tune parameters properly for other applications [22]. In this study, we proposed a new non-parametric method for similarity measurement between the motions of two mobile devices. The magnitude of acceleration and rotation are jointly represented on the same coordinate system; thus, there are no parameters to be tuned to combine the magnitudes of the acceleration and rotation. In addition, a method of reducing the effects from swinging motion of the safety hook was introduced.

III. METHOD
In this study, we propose a new similarity measurement method for safety hook fastening state recognition. We achieve this by measuring the similarity of the magnitude of acceleration and the rotation of two mobile devices in the same coordinate system. The key concept is to measure the distance between two point sets in which the magnitude of acceleration and rotation are represented in the spherical coordinate system (SCS). Hereafter, we denote the IMU-embedded mobile device attached to the safety hook of the safety harness as the mobile device of the hook and the IMU-embedded mobile device attached to the chest strap of the safety harness as the mobile device of the body.

A. ROTATION ALIGNMENT
We assume that, if the hook is fastened to the body, the transformation of both rotations would be similar. Specifically, the local coordinate systems (LCSs) of both rotations relative to each LCS at the initial time R L 0 L n at the n th sampling time would be similar. However, both R L 0 L n cannot be directly compared because the initial orientations of each mobile device are different. Therefore, a de-rotation process with respect to the initial orientation is required. IMU sensors could provide the rotation of the LCS at the n th sampling time relative to the world coordinate system (WCS) R W L n . The rotations consist of two rotations as shown in 1: 1. R W L 0 : The rotation matrix of a LCS of a mobile device at the initial time relative to the WCS.
2. R L 0 L n : The rotation matrix of a LCS of a mobile device at the n th sampling time relative to the LCS of a mobile device at the initial time.
In order to extract R L 0 L n , the rotation of the LCS at the n th sampling time relative to the WCS R W L n should be de-rotated as much as the rotation of the LCS relative to the WCS at the initial time R W L 0 .
The de-rotation can be applied by multiplying the transposed matrix R W L 0 T to R W L n as, shown in 2.

B. JOINT REPRESENTATION
We measured the similarity by using rotation in addition to the magnitude of acceleration while existing studies [20]- [22] used only the magnitude of acceleration to measure the similarity of two motions. In our proposed method, the two kinematic parameters must be jointly represented in the same coordinate system to measure the similarity. Several studies have attempted to represent accelerations and rotations in the same coordinate system [25], [26]. These studies usually attempted to analyze the movement of animals by visualizing the accelerations and rotations in the Cartesian coordinate system (CCS). However, the representation on CCS is hardly applicable for similarity measurement where multi-turn situations should be considered. Figure 1 presents an example of a multi-turn situation on each coordinate system. There are three points at n th sampling time. Each point indicates a different ϕ rotation with the same magnitude of acceleration. In the CCS, the Euclidean distance between the yellow point and the blue point is shorter than that between the red point and the blue point, yet in fact the actual ϕ rotation difference between the red point and the blue point is smaller. On the other hand, the ϕ directional rotation occupies one of the dimensions of the SCS; thus, the actual ϕ directional rotation difference is reflected in the distance calculation. Therefore, it is outstanding that the distance between the red and the blue point is shorter than that of the yellow and the blue point on the SCS. In this study, we represent the motion of a mobile device as a point set on the SCS. The procedure for generating a point at the n th sampling time is as follows: A unit vector arranged on the center of X, Y, and, Z is generated on the CCS at the initial time.
The orientation of a mobile device at the n th sampling time can be represented by multiplying the rotation matrix R The orientation represented in the CCS is transformed into a representation in the SCS. This can be represented as the polar angle θ and the azimuthal angle ϕ on the SCS as shown in 5 and 6.
In the SCS, the polar angle directional rotation and azimuthal angle directional rotation represent the rotation about the vertical and horizontal planes on the surface of the ground, respectively.
In addition to the rotations, the magnitude of acceleration also represents the motion of the mobile device.   As points are recorded at every sampling time, the motion of the mobile device is represented as a point set on the SCS.

C. MEASURING SIMILARITY BETWEEN POINT SETS
In order to measure the similarity of two point sets, we adopt the Hausdorff distance [27], which is conventionally used for measuring the distance between two point sets. However, the Hausdorff distance has a drawback that of being sensitive to outliers [28], [29].
As the construction site is an extreme environment, we could expect the complexity of the motion of a worker [4], [7], [24] can induce outliers even when the hook is fastened to a part of the body or a belt of the harness. Therefore, we adopt the average Hausdorff distance (AHD) [28] to alleviate the effects from the outliers. The equation for the AHD is shown in 9 where d(p H , q B ) indicates the Euclidean norm between two points each belonging to a point set of a hook P H and a body Q B .
While the corresponding points are required to be predetermined in the AHD, the corresponding points are deterministic with respect to the synchronized sampling time in our application. Consequently, the formula for the similarity measurement is derived from the Euclidean norm as shown in 10.

D. PARTIAL ORIENTATION BIAS COMPENSATION
The problem we are focusing on in this study is similar to those in existing studies in which attempts were made to classify whether two sensors are located in the same place (person) [20]- [23]. However, there are major differences with the existing studies and ours in which the mobile devices are attached to a part of the body. That is, the mobile device of the hook is not attached but is fastened to a harness or a part of the body. Because the hook is fastened, the mobile device exhibits a swinging motion around the fastening anchor point, and the swinging motion makes the orientation of the hook to be biased. The mobile device of the body follows the motion of the body, but the mobile device of the hook follows the motion of the body as its orientation is biased even when the hook is fastened to the body. Therefore, it is necessary to reduce the influence of the orientation bias. The orientation bias does not have the same value over the entire sampling time, but it occurs partially with different values for each segment over the entire sampling time.
To compensate the partially occurred bias, the entire sampling time is required to be divided into M segments with boundary points b [m] where m = {1, 2, · · · , M }, and a registration for the two point sets is executed in the m th segment, respectively. We assumed that an additional rotation occurs when the magnitude of the angular velocity [30] of the mobile device of the hook is higher than the maximum magnitude of the angular velocity of the mobile device of the body, which can induce the orientation bias. Based on this assumption, boundary points are selected at the moments. Because all points in both the point sets are paired according to the synchronized sampling time, we adopted Kabsch algorithm [31], which is a registration algorithm for two paired point sets.
, 0 T , Because several points p H in point set P H are compensated to p HC , the formula for the similarity measurement is modified as shown in 12.

IV. EXPERIMENTAL SETUP A. DATA COLLECTION
To the best of our knowledge, there are no existing public activity datasets that incorporate the motion of a human body and a hook in construction site activities. Although the technology for recognizing the fastening state of the safety hook is  important for the prevention of FFHs, related studies have not been actively conducted. Therefore, we collected dataset not only to evaluate the effectiveness of the proposed similarity measurement method, but also for future research on FFH prevention. In the data collection process, a total of 10 adult participants (mean age: 26.1 years, and standard deviation: 3.18 years) were asked to execute common construction work tasks. Data collection was carried out at a scaffold test bed that replicated the real construction site as closely as possible in Figure 3a.
For the experiment, two mobile devices each containing a single IMU sensor were used. For the mobile device, EBIMU24GV4 was used, which incorporates an accelerometer, a gyroscope sensor, and a magnetic sensor. Tri-axis acceleration, angular velocity, and quaternion data were collected at a sampling frequency of 250Hz from the mobile device. An RF receiver, EBRCV24GV4, 2.4GHz, was connected to a PC in which the collected data was saved. As for the attachment location of the two mobile devices, one of them was attached to the neck of the safety harness closer to the hook and the other one was attached to the chest strap of the safety harness. The exact locations are presented in Figure 3b.
In the data collection, seven construction site activities were conducted; these are presented in Figure 4. Each task consisted of a combination of construction site workers' activities [12], and these were confirmed by professional experts at the construction site. The data collection process was approved by the Institutional Review Board of Korea Advanced Institute of Science and Technology. For the task scenarios, participants were asked to execute four uniform (A,B,C, and G) and three multiple task scenarios (D,E, and F). Each uniform and multiple task scenario was performed for 64 and 128 seconds respectively. Overall, seven hours of construction site work activity data were collected. The collected dataset can be requested via https://sites.google.com/view/shfsr2022/.
The seven task scenarios can be divided into three main categories. Walking & Climbing consists of tasks that require the worker to (A) walk on a safety platform that is installed on the scaffold, (B) walk on an unstable safety platform or on a pipe, or (C) climb up or down a ladder. Assembling/Disassembling Scaffold is largely comprises tasks related to the scaffoldplatform work. Task (D) involves assembling, transporting, and disassembling scaffolding metal planks. Likewise, task (E) involves assembling, transporting, and disassembling scaffolding pipes for scaffold safety rails. Tasks performed in both (D) and (E) are performed over three sequential operations continuously within the given time. The last main task scenario is related to formwork. Task (F) involves assembling/disassembling of the Euroform; this requires the participants to carry the Euroform to a designated location and then assemble and disassemble the form work. The participants were trained prior to performing the tasks. For task (G), we initially aimed to replicate the Euroform removal process which is performed after the concrete has solidified.  However, due to the difficulty in replicating this process, we replaced this operation with picking the nails off a Euroform that was placed and fixed perpendicular to the ground.
The seven task scenarios were performed four times each under various safety hook fastening states: properly (pipe, rope, and anchor) or improperly (body). When the safety hook is fastened to either pipe, rope or anchor, we consider it as the proper usage of PPE; conversely, if the safety hook is fastened to the body, we consider it to be an improper usage of PPE.

B. EXPERIMENTAL DESIGN
Through the experiment, we planned to examine two factors: effective method of representing the magnitude of acceleration and the effectiveness of our proposed method compared with conventional methods in which only the magnitude of acceleration was used for the similarity measurement. In addition, we evaluated the effectiveness of our proposed method on various window sizes in order to consider real application scenarios.
The collected dataset consisted of tri-axis acceleration, angular velocity, and quaternion. In order to apply the collected data to the proposed method, the magnitude of acceleration, the magnitude of angular velocity and rotation matrix are required. For the analysis of motion from acceleration, the existing studies [20]- [22] removed the effect of gravity from acceleration acquired by the accelerometer. Likewise, we removed the effect of gravity by using the quaternion [32], to analyze the pure dynamic motion [33] from acceleration. We additionally applied a high pass filter with a cut off frequency of 1Hz in order to alleviate the effect from thermal drift error [33]. Subsequently, we extracted the magnitude of acceleration and angular velocity [30] from the SMV, as shown in 13 and 14. We then extracted the rotation matrix from the quaternion as shown in 15 at the n th sampling time.
The total number of instances in the collected dataset is 280 (uniform tasks: 160, multiple tasks: 120). The lengths of the uniform task and multiple task instances are 64 seconds and 128 seconds, respectively. We split the multiple task instance into three samples with a 50% hop size for an evaluation at the same window size of 64 seconds. Therefore, the total number of samples yielded 520 samples with window size of 64 seconds. To evaluate the performance, we set up various window sizes on a base-2 logarithmic (log2) scale 64, 32, . . . , 2, and 1. The number of samples varied from 520 samples (64 seconds) to 33,280 samples (1 second), depending on the window sizes.
In order to evaluate the effectiveness, we adopted the AUC, accuracy (Acc), sensitivity (Sen), specificity (Spe) and Youden's index (YI) as the evaluation criteria. Each criterion is a combination of true positive (TP), true negative (TN), false positive (FP), and false negative (FN).
Youden sindex = 1 2 (Sensitivity + Specificity). (19) The positive class and negative class indicate the case that PPE is being used properly and improperly, respectively. In this experiment, the number of samples in positive class and negative class is imbalanced because there are three type of cases (fastening to pipe, pope, and anchor) in the positive class but there is only one case (fastening to body) in the negative class. Therefore, YI was used as the main criterion in this experiment.

A. REPRESENTATION OF MAGNITUDE OF ACCELERATION
The existing studies in which the magnitude of acceleration has been used for recognition mainly represented the motion data as a continuous variable. Although it is possible to represent the data as an ordinal variable, when the data are represented as a continuous variable, it showed a comparatively better performance [21]. In particular, Pearson correlation coefficient (PCC) that measures the linear correlation between two sets of continuous data has been commonly used in the studies [21], [23] as a similarity measurement method. In order to measure the linear correlation through PCC effectively, there exists a prerequisite that the data should have a normal distribution. We examined whether the collected data could satisfy this prerequisite. From the test results, we have identified a rejection in the null hypothesis in which the magnitude of acceleration of both the hook and body comes from a normal distribution at the 1% confidence level through Kolmogorov-Smirnov test [34]. In other words, all the magnitude of acceleration in the collected dataset comes form non-normal distribution. Therefore, similarity measurement methods such as Kendall rank correlation coefficient (KRCC) and Spearman's rank correlation coefficient (SRCC), which do not necessarily satisfy normality would be effective for the recognition.
As shown in Table 2, the similarity measurement of the magnitude of acceleration using KRCC and SRCC showed comparatively better performance than that of using PCC. As KRCC and SRCC measures the correlation coefficient of two sets of ordinal data, we could expect that representing the magnitude of acceleration as an ordinal variable would more effective. Although the proposed method has three dimensions, only the magnitude of acceleration of the hook and body were used for the performance comparison; here, we represented the magnitude of acceleration as a continuous variable (a C ) or an ordinal variable (a O ) and then compared their performances. In this case, the similarity measurement method was simply Euclidean norm of the two one-dimensional time-series signals. Consequently, as a O (82.95%) exhibits a higher YI than a C (74.87%), as shown in  Table 2, it can be deduced that representing the magnitude of acceleration as an ordinal variable is more effective.

B. EFFECTIVENESS OF JOINT REPRESENTATION OF MAGNITUDE OF ACCELERATION AND ROTATION
In order to verify the effectiveness of the similarity measurement method of the two point sets, we compared the performance of each combination. The point set samples for each combination are presented in Figure 5.
In the overall results in Table 3, there exist three notable tendencies. First, representing the magnitude of acceleration as the ordinal variable provides a better performance. This trend was maintained even when the rotation information (R) was considered in the similarity measurement. Second, for combinations that include rotation information, combinations that compensate for partial orientation bias (C) showed better performance than combinations that did not compensate for partial orientation bias regardless of the representation of the magnitude of acceleration. Therefore, we confirmed that compensating for partial orientation bias for combinations with rotation information is more effective for the fastening state recognition. Finally, representing the magnitude of acceleration with the rotation rather than using only the magnitude of acceleration shows better performance in terms of similarity. However, not all the representation of the magnitude of acceleration with rotations showed a performance improvement. When the magnitude of acceleration as an ordinal variable a O and the polar angle θ are jointly represented (65.64%), it showed lower YI than that of using a O only (82.95%). It can be seen that using θ information without partial orientation bias is redundant. When the partial orientation bias of ϕ is not compensated, the best Acc (90.38%) is shown but this phenomenon occurs because it has the highest Sen (93.85%) in the imbalanced dataset.
Although the representation of a O only with the azimuthal angle ϕ showed the higher Acc (90.38%) than that of the representation of a O with all of the compensated rotations (87.18%), it showed a lower YI (86.92%), which was used as the main criterion, than the representation of a O with all VOLUME 10, 2022 of the compensated rotations. In particular, in the application scenario of this study, Spe is important because it is necessary to prevent FFHs by recognizing the moment that a worker is using the safety hook improperly. The representation of a O with all of the compensated rotations showed the highest YI (90.64%) and Spe (97.69%); therefore it was the most effective in this experiment.

C. PERFORMANCE EVALUATION ACCORDING TO WINDOW SIZE
Considering a scenario that the system is applied to a real construction site, the recognition result should be delivered to a worker and safety manager as quickly as possible. Therefore, the window size corresponding to the time when the recognition result outputs should be as short as possible while maintaining the performance as much as possible. In this experiment, we evaluated the value of YI that was maintained for each combination while the window size was reduced. Table 4 presents the YI for the window size of each representation combination. The last column shows the difference in YI when the window size is reduced from 64 seconds to 1 second. As the window size becomes smaller, a performance degradation can be observed in all the combinations. However, it can be observed that representing the magnitude of acceleration with partial bias compensated rotation could maintain the performance better even when the window size is reduced. In particular, as the window size becomes smaller, the representation of the magnitude of acceleration with the compensated ϕ (a C : 77.94%, a O : 78.75%) could maintain the performance better than the representation of the magnitude of acceleration with all of the compensated rotations (a C : 75.63%, a O : 76.25%). Moreover, the representation of a O with the compensated ϕ showed the highest YI (78.75%) among all the combinations for 1 second window size. Consequently, it was effective to represent the magnitude of acceleration with ϕ for the similarity measurement when the early stage of the scenario in the construction site was considered.

D. DISCUSSION
In this study, a method for recognizing whether a safety hook is fastened to a temporary structure is proposed to prevent FFHs. We verified the effectiveness of the proposed method by experimenting with a newly collected dataset. The proposed methodology is based on the assumption that the motion of the hook and the motion of the body are similar when the safety hook is fastened to a part of the body. In contrast to existing studies [20]- [22] that only focused on measuring the similarity of the magnitude of acceleration, this study proposes a new method for measuring the similarity by representing the magnitude of acceleration and the rotation in the same coordinate system.
From the results, two main factors can be identified. First, representing the magnitude of acceleration as an ordinal variable showed more meaningful results than representing it as a continuous variable. It was confirmed that the magnitude of acceleration did not satisfy normality. We expected that the unsatisfaction of normality is induced by outliers of the motions of task-related activities that are comparatively more complex than the motion of the existing studies. In Figure 5a, it can be confirmed that outliers were occurred in the magnitude of acceleration dimension, whereas the outliers were removed by representing the magnitude of acceleration as an ordinal variable in Figure 5b. Therefore, the performance would be improved by the effects from the outliers being alleviated by representing the magnitude of acceleration as an ordinal variable. In addition, in the existing studies, coherence was used as a representative method for measuring similarity because it showed effective recognition performance; in contrast, in this study, it was confirmed that the performance was relatively lower than that of other similarity measurement methods. This is possibly due to the redundant frequency components among all the frequency components for recognition [22], and the performance was degraded by averaging them equally.
Second, it was confirmed that the method of measuring the similarity by representing the magnitude acceleration with rotation in the SCS is more effective for recognition. It is notable that the performance improved when the partially occurring orientation bias was compensated. Since compensation was performed for the samples in all the states, the distance between the point sets can be small when the hook is either fastened to a part of the body, or to a temporary structure. Nevertheless, it can be observed in Figure 5a that the rotation of the hook fastened to a part of the body after compensation overlaps the rotation of the body; this does not when the hook is fastened to the temporary structure (especially the hook fastened to the anchor). This is because that the tendency of rotation of the body and the hook in the compensated segments are different even if the samples are compensated in the case where the hook is fastened to the temporary structure. Moreover, it was confirmed that it is more effective to measure the similarity by representing only ϕ with the magnitude of acceleration rather than representing both ϕ and θ with the magnitude of acceleration at small window sizes. It is expected that the rotation information in the θ direction is relatively redundant in the recognition. However, when the hook is fastened to a part of the body, valid local patterns for the recognition can occur from rotation in the θ direction, and we expect that the pattern can be extracted by using a point set based deep learning algorithm.
Finally, it can be observed that YI improves as the window size increases as shown in Table 4. Considering real scenarios, it can be implemented as follows: Just after recognizing that the safety hook is fastened somewhere via existing technologies [14], [35], motion data accumulates as the time passes from 1 second when the initial recognition result is outputted. Concurrently, reliable recognition results are incrementally outputted. In particular, it would be more effective to utilize the recognition results based on the representation of a O only with the compensated ϕ in the early stages (within 16 seconds), and the recognition results based on the representation of a O with all of the compensated rotations in the later stages (after 16 seconds). If the results indicate an improper use of safety harness consistently, the safety manager is notified in order to prevent FFHs in advance.

VI. CONCLUSION
Improper use of safety harnesses can cause workers to FFH. We proposed a new approach to prevent FFHs in advance by recognizing whether a safety hook is fastened to a temporary VOLUME 10, 2022 structure. Our assumption was that if a safety hook is fastened to a part of the body, the motion of the hook and the body will be similar compared to when the hook is fastened to a temporary structure. Based on this assumption, we collected a new human activity dataset using two IMU sensors attached to the chest strap of a safety harness and to a safety hook. Through experiments, we confirmed that representing the magnitude of acceleration as an ordinal variable with the rotation in SCS is effective for safety hook fastening state recognition. The highest performance in this study was limited to 90.64% YI because the high dimensional data were directly used to measure the similarity. It is expected that the performance can be further improved by extracting effective features from the given data applying a learning-based method such as machine learning or deep learning in the future. We expect that the proposed motion representation can be extended to other research fields in human activity recognition. Furthermore, the dataset was collected on a testbed that replicated the real construction site, and the task scenarios in the construction site were reflected. We hope that the collected dataset will be useful for future research related to the prevention of FFHs at construction sites.  He is also the Founder CEO of the EasyEndo Surgical Inc., the Chairperson of the board of directors of the Korea Institute of Robot and Convergence, and a member of the National Academy of Engineering of Korea. His current research interests include human-robot interaction, human-computer interaction, medical robots, telerobotics, and haptics. VOLUME 10, 2022