Collision-Free Speed Alteration Strategy for Human Safety in Human-Robot Coexistence Environments

This paper presents a novel real-time collision-free speed alteration strategy using a danger index and an elite real-coded genetic algorithm (ERGA) for environments in which humans and robots coexist or cooperate, in order to guarantee the safety of an operator who works with a collaborative robot. A danger index based on ellipsoid modeling of the operator and robot describes the degree of safety during human–robot interactions. The ERGA and a penalty function are used to solve the constrained nonlinear optimization problem to change the handling speed of the robot. Comparative simulation results show the superiority of the proposed method by comparing to two existing methods. The applicability of the proposed method is verified using two experiments involving a 6-DoF industrial manipulator with an EtherCAT network protocol, an RGB-D sensor and a real-time operation system.


I. INTRODUCTION
Many industrial applications require humans and robots to coexist so that the safety of human operators must be guaranteed at all times. The workspaces for robotic manipulators often overlap those for human operators and numerous conventional methods use barriers to isolate robots and avoid contact and collisions. Few studies ensure safe and efficient human-robot collaborations by relaxing the separation of robot and human workspaces [1], [2].
In such collaborations, the current state of the environment must be firstly determined. Different types of sensors, such as a floor mat [2] and tactile sensors [3], were used to dynamically detect a human operator's activity in an environment. An RGB-D sensor was affordable and flexible in terms of installation and allowed a high resolution for problems that involve computer visions and robotics [4]- [6]. Since the Microsoft Kinect sensor was released, more than two thousand papers have been published for conferences or in journals of the IEEE [4]. Flacco and Luca [7] used the depth information from two Kinect sensors to monitor The associate editor coordinating the review of this manuscript and approving it for publication was Yongping Pan . a task that involves collaboration between a human and a robot. Morato et al. [8] proposed an exteroceptive sensing framework that uses a multiple Kinect sensor to perform an assembly task that involves human-robot cooperation.
Studies by Flacco and Luca [7] and Morato et al. [8] showed that if a human operator is detected inside a robot's workspace, a collision is possible, so real-time collision detection is important for collision-free planning. Many collision detection algorithms [9] were used in robotics and for computer-aided design. However, collision detection may encounter a highly computational complexity for moving objects. In order to reduce computational complexity, bounding volumes are used to reduce the computational cost because geometric primitives are used. Ellipsoids have a simple mathematical representation so they are used as bounding volumes for the detection of collisions with natural objects [10]- [13]. Rimon and Boyd [10] used minimum-volume enclosing ellipsoids to calculate the distance between a robot and an obstacle. Bablan and Bone [13] used spheres to represent the distance between a robot and a human and used this distance to determine a cost function.
The level of danger to human operators in an environment that involves human-robot must be calculated and this measurement is used to formulate a safety-based constraint for a control strategy. If a human operator moving toward a working robot is a highly dangerous scenario, the robot must operate at a lower speed or stop [14]. Many studies quantitatively defined the safety of human operators in environments that feature human-robot collaboration. The potential field approach that was proposed by Khatib [15] converts the distance between the current position and the goal position for an end-effector into a vector field to generate an appropriate control torque, so torque commands require an exact model of the manipulator and these are often not available because variations in friction, mass and inertia may be unknown. Polverini et al. [16] presented a safety evaluation for human-robot interactions that use a kinetostatic danger field to capture the configuration and velocity of the complete robotic manipulator to define a safety-oriented control strategy. However, these methods only consider current states [15], [16], the algorithms become trapped in local minima and do not generate a smooth trajectory. Optimized path planning was used by Hwang et al. [12], using a collision-trend index that is defined by projecting the ellipsoids onto a Gaussian distribution and a penalty index to change the robot's speed along a predefined path. Kulic and Croft [17] also established a strategy to make human-robot interaction safer by minimizing the danger criterion, which is calculated in terms of the relative velocity and the relative distance between the human worker and robot. Tsai et al. [18] proposed a safety index that is utilized for the inequality constraint to ensure the safety of a human worker within a shared workspace.
Optimization-based path planning problems that involve an equality or an inequality constraint are defined as constrained optimization problems. A genetic algorithm (GA) with a penalty function has been widely used to solve constrained nonlinear optimization problems [19] because a GA is superior to gradient methods and is simple and easy to implement [20]. However, a traditional genetic algorithm uses a binary string to represent a solution such that a binary-coded GA is not suited to a continuous search space with high dimensions or to high-precision numerical problems. A real-coded GA [21] represents the optimization variables using floating-point numbers. Tsai et al. [22] showed that an elite GA is superior to a non-elite GA because a non-elite GA converges prematurely and the solution can be worse in the next generation. The study [22] proposed an elite GA that uses a reproduction strategy with an elite policy, which is more effective for many global problems.
Cooperation or collaboration between humans and robots requires real-time computations and a fast response time. EtherCAT (Ethernet Control Automation Technology) is an industrial Ethernet protocol that retains the merits of Ethernet, such as an industrial field-bus protocol with a 100 Mbps transfer speed, a short response time, a short cycling time and accurate time synchronization. EtherCAT also allows a network to contain several hundreds or thousands of I/O devices without any hardware modification so that EtherCAT is widely used for real-time control systems and in industrial automation domains [23].
Jerk is the time derivative of acceleration and is a feature of motion control for a robotic manipulator. The amplitude of the jerk is related to joint wear, trajectory tracking performance and psychological stress in a human operator in an environment that features collaboration between humans and machines. Large jerk values increase wear and reduce the life span of manipulators such that minimum-jerk path planning is essential for robotics studies. Goor [31] proposed a smooth path planning method for human-centered and predictable robots that features a minimum jerk strategy. Chen et al. [32] proposed multi-level simultaneous minimization at the jerk level for jerk-bounded robotic motion that increases the safety of humans in an environment that involves human-robot interaction. Rojas et al. [33] proposed a scheme that generates a physically and psychologically safe minimum-jerk trajectory for collaborative assembly stations. This minimum-jerk trajectory mitigates against joint errors, vibration and wear to robotic manipulators during the trajectory tacking phase and then increases the degree of confidence for a human operator.
Confidence is significantly different to safety. Even if a robot moves safely, an operator may not have confidence in the safety of operations. The authors in [37] showed that a path planning must generate a collision-free path and avoid negative emotional responses in a human operator. A similar study of human proxemics preference in terms of the attributes of robots was presented by Bhagya et al. [38].
Motivated by these studies [7], [8], [10], [12], [18], [21]- [23], [33], [38], this paper proposes a real-time strategy to change the collision-free speed for robots that work in an environment where there are humans using ellipsoid modeling to ensure the safety guarantee of the human operator, and determines its effectiveness by simulations and experiments. A comparison with two current methods [8], [12] shows that the proposed method allows more effective real-time changes to the collision-free speed for environments in which humans and robots coexist using a danger index and a real-time ERGA, in order to ensure the safety of a human operator who works with an industrial collaborative robot. The proposed method may provide useful references for professionals working in the field of human-robot coexistence or human-robot collaboration.
The remainder of this paper is organized as follows. Section II describes the scheme for collision detection using ellipsoid-based modeling and a danger index quantitatively measures the degree of safety of human-robot collaboration. Section III elaborates the proposed speed alteration strategy which is formulated as a constrained nonlinear optimization problem to prevent collisions in an environment in which humans and robots coexist. Section IV describes simulations that show the effectiveness and the superiority of the proposed method in comparison with two current methods [8], [12]. Section V details the experimental results that show the practicability of the proposed method. Section VI concludes this paper. VOLUME 8, 2020

II. HUMAN-ROBOT MODELING AND COLLISION DETECTIONN
In this section, the shapes of a human and a robot are modeled using ellipsoids. A danger index with ellipsoids is used to quantitatively measure the degree of safety for humans to detect collisions quickly.

A. HUMAN-ROBOT MODELING
Modeling the shape of humans and robots is necessary for fast and robust 3D collision detection algorithms. Moving objects increase the computational complexity so ellipsoids are used to model the shape of humans and robots to rapidly detect a collision. For robot modeling, any point on the link of robot is represented by where X = (X , Y , Z ) is the position vector for a point on the ellipsoid for link i with respect to the robot base coordinate system and x is the position vector for an elliptic equation with a parametric form. The homogeneous matrix A 0 i−1 denotes the relative position of the center of link i − 1 with respect to the base frame in the form: where n = (n x , n y , n z ), o = (o x , o y , o z ) and a = (a x , a y , a z ) respectively represent the normal vector, orientation vector and approach vector, and p = (p x , p y , p z ) denotes the positions of the end-effector in link i − 1 and Rot (z, θ i ) is the rotational transformation through angle θ i along the z axis.
is the translation from joint i to the center (X ci , Y ci , Z ci ) of link i. From (1), one obtains so the standard elliptic equation for link i is To model a human, an RGB-D sensor is used to acquire the skeletal joints of a human operator in the 3D space from Microsoft Kinect sensing framework. Most collisions occur with the upper part of the human body, so this study only models the upper part using ellipsoids. To simplify the human model, the homogeneous transformation and rotational transformation are ignored, so A 0 i−1 and Rot (z, θ i ) are both identity matrices.

B. COLLISION DETECTION USING A DANGER INDEX
The links for a human operator and a robotic manipulator are modeled as ellipsoids using (5). These ellipsoids are used to determine possible intersections to detect a collision. The complexity of collision detection increases exponentially with the number of ellipsoids. Hwang et al. [12] proposed a fast method to detect a collision between ellipsoids using a geometric transformation, which maintains relative geometric equivalence and reduces the computational complexity of determining the overlap between two ellipsoids. This method deforms an ellipsoid into a point between two ellipsoids using an algebraic transformation. Collisions between ellipsoids are more simply detected by determining whether the point, (0,0,0), is located outside or inside the ellipsoid. The algorithm to detect a collision between the links of a human operator and a robotic manipulator is written as: if F ij x , y , z ≤ 1, then one collision occurs (6) where z is the transformed ellipsoid for link i of the robot with respect to the coordinate frame for link j of the human.
The danger index (DI) is the reciprocal of formula (6) and gives a quantitative value for the degree of safety for a human operator.
If a collision occurs, the danger index is greater than or equal to 1.

III. STRATEGY TO CHANGE SPEED AND DETERMINING THE OPTIMAL SPEED
This section details the proposed strategy to change the speed, which is formulated as a constrained nonlinear optimization problem for an environment in which humans and robots collaborate. The ERGA and a penalty function are used to solve this optimization problem to guarantee the safety of a human operator.

A. SPEED ALTERATION STRATEGY
In a dynamic environment, path planning is decomposed into two sub-problems, as proposed by Kant and Zucker [24]. The path of the robot is planned to allow it to avoid static obstacles and the velocity of the robot is planned to avoid uncontrollable moving obstacles. In some circumstances, it is not possible to change the speed of the human operator but it is easy to change the velocity of the robot. The proposed strategy changes the speed of the robot to prevent collisions using a danger index. The robot is assumed to move along a preprogrammed path at a variable speed. If a collision is 80122 VOLUME 8, 2020 predicted, the robot slows or waits to reduce the likelihood of injury. The preprogrammed speed of robot is defined as: where T denotes the total elapsed time and the triple pairs; (R x_g , R y_g R z_g ) and (R x_i , R y_i , R z_i ) respectively represent the goal and initial positions of the end-effector of the robot along the x, y and z axes. Information about the human operator and robotic manipulator is estimated to predict a collision using a constant velocity model to estimate the center positions of the ellipsoids. The pose of the ellipsoid centers is transformed from current and previous sensor measurements and is used to compute the velocity of the moving ellipsoids in the k-th sampling instant as: where P (k) and P (k − 1) are the current pose and the previous pose of the ellipsoid center and t is the sampling interval. Using the current pose P (k) and velocity V (k), the estimated pose of ellipsoid is calculated by Remark 1: The preprogrammed speed in (8) is calculated using Hwang's method [25], but other motion planning methods are possible [26]. Indeed, (9) is a simple method to estimate the pose using the method in another study [18]. The method in (9) is used for this study to determine a onestep-ahead pose prediction. Measurement noise is eliminated or reduced by a subtraction operation. The experimental results confirm that estimating the pose one-step-ahead is effective, so measurement noise does not affect the prediction. However, if the signals from the RGB-D sensor are significantly corrupted with measurement noise, a Kalman filter [8] filters noise such that (9) is used to calculate a future one-step-ahead pose estimate more accurately. The speeds of the robot end-effector and tool center point can be changed. A system that guarantees the safety of a human operator in an environment in which humans and robots collaborate must comply with ISO 10218-1 safety standards, which specify the maximum speed of a robot that allows safe working during collaboration with a robot. This speed cannot exceed 250mm · s −1 . The optimal speed of the manipulator is determined by minimizing the following objective function: where DI ij X (k + 1) is the danger index between link i of the robot and link j of the human operator at sampling instant k + 1; V preprog and V select are the preprogrammed speed and selected speed of the robot, respectively; P is a parameter that allows the selected speed to approach the preprogrammed speed and D is a smoothing parameter that decreases the variation in the selected speed to allow the robot arm to move smoothly. This objective function (11) constitutes a constrained nonlinear optimization problem, which ensures that the danger index has a value of less than 1, so the velocity of the end-effector of the manipulator that avoids a collision is planned by minimizing the objective function.

B. DETERMINING THE OPTIMAL VELOCITY USING AN ERGA
The proposed optimal speed that avoids collisions is formulated as a nonlinear constraint problem with inequality constraints. A penalty function is used to ensure compliance with the two inequality constraints. F obj (V select (k)) is defined as the sum of the objective function f obj (V select (k)) and the penalty term, which depends on the constraint violation DI ij ( X (k + 1)), as shown in (12). If the constrained condition is greater than or equal to 1, the danger index has a constant value. If the constrained condition is less than 1, the danger index is zero. The optimization algorithm is reformulated as: where R is a positive and real penalty parameter. The ERGA is then used to solve the constrained nonlinear function in (11) for two constrained conditions: (i) the danger index must be less than unity to avoid collisions between the industrial robotic manipulator and human operator and (ii) the speed of the industrial robotic manipulator must not exceed 250 mm · s −1 to comply with the ISO 10218-1 safety standard. The optimization strategy minimizes the objective function (11) and the selected speed to approaches the preprogrammed speed to reduce variations between the current and neighboring speeds. The optimal speed alteration algorithm is described as follows; two individuals in the current population with lower fitness values are chosen as the elite, the crossover operation is selected using a uniform arithmetical scheme, and the mutation operation is random. Fig. 1 shows the pseudo-code of the ERGA for the proposed speed alteration strategy. If a human operator is detected by the Kinect vision sensor, the danger index for each joint of the human and robot is calculated using the ellipsoids that are generated by modeling. The ERGA features an initialization phase, elite selection, crossover, mutation and calculates the fitness of the penalty function. The algorithm is terminated when the 100 th generation is completed because the objective function usually converges to its minimum after the five generations, so 100 generations are sufficient for the proposed algorithm to converge to the optimal velocity at the sampling instant, as shown in Figs. 11 (c) and (d). The optimal output for the ERGA is sent to control the velocity of the robot's end-effector. Furthermore, if the human operator is not in the workspace, the robot moves at the preprogrammed speed.

IV. SIMULATIONS AND DISCUSSION
A rigid 6-DoF industrial robotic manipulator, which is called RA605 and is made by the HIWIN Company, is used to determine the effectiveness of the proposed strategy for an environment that contains humans and robots. As shown in Fig. 2, each link of the robot and the upper human body is modeled by ellipsoids. In the simulation, the right hand of the human operator is placed into the scheduled path of the robot manipulator and the hand collides with the robot (marked with red color). The radius of the ellipsoids for the right hand and the end-effector is 50 mm.  Fig. 3(a) shows the positions of the end-effector when the robot moves along the positive direction of the y axis. Fig. 3(b) shows the positions of the human right hand as it initially moves from (x, y, z) = (400 mm, 140 mm, 620 mm) to (x, y, z) = (400 mm, 140 mm, 700 mm) at a velocity of 100 mm · s −1 from the beginning to the 9 th sampling instants. From the 10 th to 12 th sampling instants, the right hand remains at the same position for 3 sampling instants and then moves back to the initial position from (x, y, z) = (400mm, 140mm, 700 mm) to (x, y, z) = (400 mm, 140 mm, 620 mm) at a velocity of −100mm · s −1 between the 12 th and 20 th sampling instants. The operator's right hand stops at The proposed speed strategy is then applied to avoid collisions. The ERGA is used to solve the constrained nonlinear function as in (12), where the initial population is set to 50 and the respective probabilities of crossover and mutation are 0.8 and 0.1. The algorithm is simulated using the MATLAB program. The speed of the robot is controlled using the proposed method to avoid collisions. The values for the velocity parameter (P), the smoothing parameter (D) and the penalty parameter (R) for the objective function are assigned using the following three simulations.
The value of the penalty parameter (R) is chosen by simulations using a fixed value for the velocity parameter (P) and the smoothing parameter (D) of 1. For values of R of 1, 10, and 100, Fig. 5(a) shows that R is too small to allow the penalty function to have any significance. For a value for R of 10, the danger index remains less than 1 if the speed of the robot is changed. The result for a higher R value (R = 100) is similar to that for R = 10. In Fig. 5(b), the velocity profile  for R = 10 is smoother than that for R = 100, so the value of R is set to 10 to avoid collisions.
The value of the velocity parameter (P) is selected using simulations with a penalty parameter (R) and a smoothing parameter (D) have respective fixed values of 10 and 1. For value of P of 1, 10 and 100, Fig. 6(a) shows that P is too large so the penalty function has no effect and a collision occurs (the danger index is more than 1) for a value of P of 100. However, the velocity profile is smoother for lower values of P(= 1) in Fig. 6(b). A P value of 1 ensures a smooth and safe path.
The value of the smoothing parameter (D) is chosen using simulations with respective fixed values for the penalty parameter (R) and the velocity parameter (P) of 10 and 1. Let D be set to 1, 10 and 100, and Fig. 7(a) shows that if the  value of D is too large, the penalty function has no effect. Obviously, the danger index is larger than 1 if D is 100. Fig. 7(b) illustrates that the velocity profile is smoother as the value of D increases, but a longer run time is required for the robot to reach the goal position.
These results show that the parameter set, P = 1, D = 1 and R = 10, allows collision-free and smooth path planning. The evolutions of the resultant danger index for the proposed approach are shown in Fig. 8(a), which shows that the danger index is not greater than 1 such that the proposed method prevents collisions between the human operator and the robotic manipulator. Fig. 8 (b) shows the changes to the velocity profile for the robot that prevent collisions with the human operator. To allow the robotic manipulator to reach its goal pose, the danger index gradually increases from 0.7596 to 0.994 during the 20 th to the 29 th sampling instants so that after 20 th sampling instant, the operator's right hand stays at the same position (x, y, z) = (400 mm, 140 mm, 620 mm), near  the goal pose (x, y, z, Yaw, Pitch, Roll) = (368 mm, 120 mm, 668.5 mm, 0 • , 0 • , 180 • ) for the robotic manipulator. At the 29 th sample, the manipulator safely achieves its target pose using the proposed method.

B. COMPARATIVE SIMULATIONS
Two simulations will show the superiority of the proposed method over to two existing methods [8], [12]. The objective function for Hwang's method in [12] is described as (13) where P ij (k)i s the collision-trend index between link i of the robot and link j of the human operator at sampling instant k and the second term of (13) is used to force the robot to move at the selected speed to reduce the sum of the collision-trend index. Finally, V preprog and V select are the preprogrammed speed and the selected speed for the robot, respectively. The proposed method is also compared with the bi-modal control strategy of Marato et al. [8]. The bi-modal control strategy moves the manipulator at a specified speed if the relative distance between it and an operator exceeds a threshold distance; otherwise, the manipulator is stopped.
The first simulation compares the proposed method, Hwang's and Morato's methods using simulations settings that are identical to those described in Section 4.A. Fig. 9(a) shows that the danger index for the proposed method is lower than that for Hwang's method, so the proposed method is safer than Hwang's method. Fig. 9(b) shows that the proposed method produces smoother selected velocities than Hwang's and Marato's methods do.
The previous study [27] defines the properties of smoothness in terms of speed, acceleration and jerk.  Fig. 9(d) and Table 1 show that the standard deviations for jerk for Hwang's and Morato's methods are respectively 201.87% and 944.07% of that for the proposed method. The results for mean and standard deviation of the jerk show that the proposed method outperforms Hwang's and Morato's methods [27]. The life span of a manipulator is increased because the speed alteration method avoids damage to the robot motors, reduces mechanical wear on the robot joints and increases confidence for human-robot collaboration and accuracy.
The second simulation specifies parameter settings of P = 1, D = 1 and R = 10 for the objective function. The preprogrammed velocity for the robot is negative as shown in Fig. 2(d)-(f). The robot moves from its initial pose (x, y, z, Yaw, Pitch, Roll) = (368 mm, 0 mm, 668.5 mm, 0 • , 0 • , 180 • ) to its goal pose (x, y, z, Yaw, Pitch, Roll) = (368 mm, -120 mm, 668.5 mm, 0 • , 0 • , 180 • ) at a preprogrammed velocity of −50mm · s −1 for a total elapsed time of T=2.4 s and a sampling time interval of t = 0.1 s is used. Fig. 10(a) shows the positions of the end-effector when the robot moves backwards along the y axis. Fig. 10(b) shows the positions of the human right hand as it initially moves from (x, y, z) = (400 mm, −140 mm, 620 mm) to (x, y, z) = (400 mm, −140 mm, 700 mm) at a velocity of 100 mm · s −1 from the beginning to the 9 th sample. During the 10 th to 12 th samples, the right hand remains in the same position for the three sampling instants and then returns to its initial position from (x, y, z) = (400 mm, −140 mm, 700 mm) to (x, y, z) = (400 mm, −140 mm, 620 mm) at a velocity of −100 mm · s −1 between the 12 th and 20 th sampling instants. Finally, the operator's right hand stops in its original position after the 20 th sampling instant. Fig. 11(a) shows that the danger indexes, whose values are close to unity, are respectively 0.9979, 0.9997 and 0.995. All danger indexes in Fig. 11 are less than unity, so a human operator's safety is guaranteed and there are no collisions. Fig. 11(b) shows that the selected velocity for the proposed method is close to the programmed speed of −50mm · s −1 . The selected velocity for Hwang's method [12] is the maximum speed. The velocity obtained for Hwang's method changes abruptly to −250 mm · s −1 at the ninth sampling instant, and abruptly again to 250 mm · s −1 at the tenth sampling instant. For the proposed method, the selected velocity is −49.88 mm · s −1 at the same 9 th sample and this jumps to  Fig. 11 (f) shows that the respective standard deviation of jerk for Hwang's and Morato's methods is 3875.49% and 223.75% of that for the proposed method.
The simulation results in Figs. 8, 9, 11 and Table 1 show that the proposed method is superior to the bi-modal control strategy and Hwang's method in that it allows smoother speed changes, produces less jerk, allows safer working when there are short distances between humans and robots [34]- [36] and faster movement from the initial pose to the destination pose.

V. EXPERIMENTAL RESULTS AND DISCUSSION
Experiments using an EtherCAT-based six-degrees-of freedom (6-DoF) industrial robotic manipulator show the practicability of the proposed method. Figs. 12 and 13 show the experimental setup, which uses a 6-DoF industrial robotic manipulator (HIWIN Inc., RA605) with 6 motor drives (Panasonic Inc., A6SF), an industrial computer controller (NEXCOM Inc., 3600E) with a real-time operation system (IntervalZero Inc., RTX64) and an RGB-D sensor (Microsoft Inc., Kinect v1). The control kernel interacts directly with the 6 motor drives via the EtherCAT protocol. The Integrated development environment with Microsoft Visual Studio C# is used to code the application programming interface.
To show the effectiveness and applicability of the proposed ERGA, two experiments used a population of 100, a crossover probability of 0.8, and a probability of mutation of 0.1. Below were the parameter settings: the sampling time t = 0.1 s, the penalty parameter R = 10, the velocity parameter P = 1 and the smoothing parameter D = 1. The proposed method was implemented on an i5-CPU industrial computer with a 2.7 GHz clock rate to perform online computations on images that are captured at 10 frames per second (FPS).

A. EXPERIMENT 1
Figs. 14(a) and (b) show the experimental scenario in which a human operator walked close to a RA605 manipulator and the right hand of the human operator moved up to the manipulator. In Fig. 14(c), the right hand of the human operator entered the planned path of the manipulator for a period of time. In Fig. 14(d), the right hand moved away from the robot. Fig. 15(b) depicts the paths of the right hand, as captured by the Kinect vision sensor. In Fig. 15(a), the robot moved from the initial pose (x, y, z, Yaw, Pitch, Roll) = (368 mm, 100 mm, 668.5 mm, 0 • , 0 • , 180 • ) to the goal pose (x, y, z, Yaw, Pitch, Roll) = (368 mm, −100 mm, 668.5 mm, 0 • , 0 • , 180 • ) at a preprogrammed velocity V preprog = −50mm · s −1 and V preprog = 50mm · s −1 ,and repeatedly moved forward and backward from the initial pose to the goal.
From the start-up to the 42 th sampling instant, the human operator was outside the detection range of the Kinect sensor. The control method was not activated, so the selected speed was the same as preprogrammed velocity, as shown in Fig. 16(b), and the danger index was equal to zero in Fig. 16(a). Fig. 16(b) shows the changes in speed and direction for the proposed method. After the 42 th sample, the human operator was within the detection range of the vision sensor and the proposed method was activated such that the danger index was not equal to zero. The danger index increased when the human operator walked close to the robot. If the danger index was less than 1, the proposed method successfully planned the velocity of the robot, such as at the 80 th sample. At the 151 th sample, if the proposed method was not used, the robot moved with a preprogrammed velocity of 50 mm · s −1 . One collision then occurred because the danger index was 1.419. Using the proposed approach, the velocity was decreased to -5.9mm·s −1 so the collision was avoided and the profiles of the danger index were less than 1. The danger index was 0.8657 at the 151 th sample. After the 151 th sampling instant, the smoothing parameters ensure that the selected speed smoothly approached the preprogrammed velocity. The results show that the proposed method successfully plans a smoother velocity for the robot and ensures the safety of the human because there are no collisions when the human and robot collaborate.
Remark 2: To stop the robot and avoid a collision with the human, a preprogrammed speed of 0 can be used. The optimal output for the ERGA stops the robot if the selected velocity is 0.

B. EXPERIMENT 2
An object pick and place task is shown in Figs. 17(a), (b) and (e). There is a RGD-D sensor behind the robot, an electric gripper on the end-effector of robot and an object on the table to be grabbed into the basket. Figs. 17(c) and (d) show a collision scenario for this pick and place task as the human operator walked to the robot and the hand of the human operator was close to the basket (in the pre-planned path of the robot) for a period time. The hand of the human then moved away from the robot before the robot placed the object. Fig. 17(f) shows the human operator picking up the basket after the robot finished its task. Fig. 18(a) shows the robot moving from the initial position (x, y, z) = (594.5 mm, -266 mm, 404.9 mm) to the intermediate position (x, y, z) = (594.5 mm, −266 mm, 105 mm), (x, y, z) = (594.5 mm, −266 mm, 404.9 mm), (x, y, z) = (594.5 mm, 174 mm, 404.9 mm), (x, y, z) = (594.5 mm, 174 mm, 105 mm), and then the goal position (x, y, z) = (594.5 mm, 174 mm, 404.9 mm) at a preprogrammed velocity of V preprog = -100 mm · s −1 , V preprog = 0 mm · s −1 and V preprog = 100 mm · s −1 for the pick and place task. From the 32th to 36th sampling instant, the robot grabbed the object using an electric gripper. At the 141 th sample, the robot placed the object into the basket. Fig. 18(b) shows the path of the human hand as captured by the vision sensor.
Figs. 19 (a) and (b) show that the human operator was outside the detection range of the vision sensor from the beginning until the 87 th sampling instant and the control approach was not used so the selected speed remained as the preprogrammed velocity and the danger index was zero. After the 87 th sampling instant, the human operator was within the detection range of the vision sensor and the proposed method was used so that the danger index increased when the human operator walked to the robot. At the 105 th sample, the robot moved at a preprogrammed velocity of 100 mm · s −1 without the proposed method. One collision occurred since the danger index was 1.094. Using the proposed approach, the velocity was reduced to 30 mm · s −1 such that no collision occurred and the profiles for the danger index were less than 1. The danger index was 0.744 at the 105 th sampling instant, and, afterward, the proposed method allowed collision-free and smooth changes in speed. The experimental results show that the proposed method plans a collision-free and smooth speed for the pick and place task that involves a human.
The computation time for the proposed method was about 0.085 s, so a control action was initiated within the sampling interval of 0.1 s via the EtherCAT network. A study by Zhang et al [28] showed that a human's reaction time is 0.25 s, so the machine must capture images at more than 4 FPS to detect reactions to a robot's action. The proposed method is 2.5 times faster than the reaction time of a human. The sampling rate for the proposed method was 10 FPS and the EtherCAT control network had a 100Mbps transfer speed and short response time and short cycling time so the proposed one-step-ahead prediction scheme using the real-time operation system, RTX, was sufficient to detect human reactions, in order to avoid collisions.

C. USERS' EVALUATION
A questionnaire (''Did you feel comfortable?'') was used to measure the emotional responses of those who participated in both experiments. To avoid harm to the participants, all were informed about the details of the experiment before it began. The participants were aged between 30 and 50 years (M = 41.2, SD = 6.5). Comfort level was recorded using a 5-point Likert scale (1 = very uncomfortable, 2 = uncomfortable, 3= neutral, 4 = comfortable, and 5 = very comfortable) for all the 20 participants. The interview determined the participants' reaction to interaction with and without the proposed method. The psychological  This experiment was also used to show the practicability of the proposed approach in terms of human robot-proxemics. The mean and standard deviation values for human-robot proxemics distance for all 20 participants are shown in Fig. 21, where the proposed method produced the lowest mean value, which shows that participants were willing to walk close to a robot if a robot had reliable collision-free path planning for human-robot interactions. The t-test statistic was then employed to evaluate the significance of the effects on human-robot proxemics distance with the proposed method and the robot without the proposed method. From the t-test results, the p-values for Experiments 1 and 2 are 1.07 × 10 −19 and 5.24 × 10 −21 , respectively. The results for the emotional responses and human robot-proxemics of participants in these experiments show that the proposed method is reliable.
This experiment also revealed that occlusion is a common problem for vision-based methods. The use of multiple vision sensors in the workspace minimizes this problem. If there is insufficient information from one camera about a human operator, another camera can be used to capture information that is occluded. Some studies [7], [8], [39] used multiple RGB-D vision sensors to address the problem of occlusion. However, the use of the multiple vision sensors results in data fusion problems and camera calibration problems [29], [43], [44]. A Kalman filter method [40] was used to address the problem of fusion for multiple RGB-D sensors for a system that tracks a human skeleton and Wu et al. [41] proposed a toolkit for tracking occluded human joints that merges the field of view of multiple RGB-D sensors and uses geometric calibration and affine transformation.

VI. CONCLUSION
The paper has presented a collision-free speed alteration strategy for environments in which humans and robots coexist. The danger index uses ellipsoids to model the robot and human operator, in order to measure the safety of a human operator during human-robot interactions. The elite real-coded genetic algorithm (ERGA) with a penalty function is used to solve a constrained nonlinear optimization problem in real time. The simulation results show that the proposed method prevents better collisions between a collaborative robot and a human operator than two existing methods [8], [12] do. The experiments using a 6-DOF industrial manipulator with the EtherCAT protocol and an RGB-D sensor has demonstrated the effectiveness and practicability of the proposed method.
Although the proposed method has planned onedimensional moving paths, it can be easily extended to nonlinear movements by following many piecewise linear paths similarly to another study [30], in which a nonlinear smooth path is interpolated or synthesized using many piecewise linear paths. Future studies will involve more advanced scenarios to change the velocity and direction along a fully nonlinear path. In practical terms, a slower velocity (250 mm · s −1 ) reduces the productivity for a task that involves human-robot collaboration [42] so the algorithm will be extended to less constrained conditions to increase productivity. To get rid of the speed limitation (250 mm · s −1 ), the proposed speed alteration strategy with the braking time capabilities of robot remains an important and challenging direction for future work.