A Muscle Teleoperation System of a Robotic Rollator Based on Bilateral Shared Control

The approach that achieves the teleoperation between human muscle signals and the mobile robot is increasingly applied to transfer human muscle stiffness to enhance robotic performance. In this paper, we develop a mobile rollator control system applying a muscle teleoperation interface and a shared control method to enhance the obstacle avoidance in an effective way. In order to control intuitively, haptic feedback is utilized in the teleoperation interface and is integrated with EMG stiffness to provide a large composition force. Then the composition force is implemented with an artificial potential field method to keep the robotic rollator away from the obstacle in advance. This algorithm is superior to the traditional APF algorithm regardless of the required time and trajectory length. The experimental results demonstrate the effectiveness of the proposed muscle teleoperation system.


I. INTRODUCTION
Muscle computer interface (MuCI) has attracted growing research interest in various research fields such as advanced robotic hand [1], exoskeletons [2], and mobile robot [3] etc.. Advances in MuCI have provided noninvasive and stable access to muscle activity. Therefore, the muscle teleoperation system establishing a connection between robotic systems and humans can be developed. The motion intention can be reflected directly from the muscle teleoperation system, as shown in Fig. 1. Muscle teleoperation technology offers guarantees to assist older people in regaining independence and performing tasks beyond their physical capabilities. Combining with advanced intelligent control methods, shared teleoperated control can not only achieve precision but also enhance the performance.
In the aging society we are facing now [4], with the decline of the mobility of the elderly, it is very crucial to develop assistive devices to enhance mobility. Multiple assistive devices such as canes, walkers, rollators are developed focusing on mobility support. The person's remaining locomotion capability is applied to move with the assistive The associate editor coordinating the review of this manuscript and approving it for publication was Zhiwu Li . devices. However, these assistive devices still do not apply to patients with severe injuries, especially those with lower limb disorders. To overcome these drawbacks, robotic rollators are developed as a new, emerging assistive device due to their advantages of intuitive operation, semi-automation, and automation.
Electro-biological signals are widely used in the human-robot interaction (HRI). Electromyography (EMG) is considered as one of the most effective and oldest electrobiological modalities. In the 1920s, Edmond Jacobson used EMG for clinical research in progressive muscle relaxation therapy technology [5], [6]. Neuromuscular activity generates EMG signals ranging from 100 µV to 90 mV [7]. When designing a HRI teleoperation system to connect mobility dysfunction with a robot, the EMG signals have typical behaviors as an important feature to be considered. A force estimation algorithm is presented in [8] to establish a relationship between the force of hand and the sEMG signal. Reference [9] presented a noninvasive electroencephalogram (EEG) computer interface to move a mobile robot between different rooms with a performance ratio of 0.74 compared to manual control. A brain-computer interface applying EEG signals is proposed in [10] applying a shared-control strategy. However, the signal level of EMG signals is higher than EEG signals and other electro-biological signals, which makes it easier to be distinguished. It is interesting to apply the EMG perception mechanism to enhance the intelligence of the telerobots [11], [12]. Reference [13] presented an EMG-based admittance control method for an exoskeleton robot. A brain-computer interface utilizing the EEG signals is proposed in [14] to control a mobile robot remotely and in corridor environments [15]. A mobile robot control method using EMG signals as robot command is proposed in [16]. EMG not only can be utilized to control command but also has the ability to detect the intention of the person as feedback control. In our previous work, a novel design of robotic rollator and the trajectory tracking controller are presented in [17] and the trajectory tracking controller based on neural approximation is proposed in [18]. In [19], EMG signals are combined with depth vision learning to recognize hand gestures automatically. The bilateral teleoperation control of a redundant manipulator is proposed in [20] to achieve haptic feedback on the end effector.
Obstacle avoidance is considered as a vital characteristic of mobile robots in unstructured environments. A modified artificial potential field algorithm is represented in [21] for obstacle avoidance to solve the local minimum problem. Artificial neural network and APF are combined in [22] to control path planning of an autonomous multi-wheeled combat vehicle in real time. Although APF can achieve good performance in the above literatures, it is designed from a human perspective. It is possible to cooperate ''passively'' with human teleoperator while doing the obstacle avoidance.
Shared control technique has an effective impact on the performance of robots controlled by human partners. An adaptive shared control method is proposed in [23] to control assistant wheelchair applying EEG signals. An EMGcontrolled prosthetic hand of the shared control is developed with different ratios [24]. However, these shared control methods utilizing in robotic control are not integrated with haptic feedback devices. It is interesting to combine the EMG active teleoperation with haptic feedback devices in a shared teleoperation control interface to achieve obstacle avoidance.
In this paper, we proposed a teleoperation control system combining EMG signals based on shared control to control the robotic rollator in the unstructured environment with obstacles. Firstly, in order to enhance the intuitive operation performance, the force feedback is reflected in the teleoperator presenting the distance between the robotic rollator and obstacle. The teleoperator can react according to the force feedback. Then the reacting EMG stiffness is estimated using MYO armband. Afterward, the shared control method is adopted to change the robotic rollator's trajectory when the obstacle is detected. Finally, the EMG stiffness is utilized as a virtual fixture to improve teleoperation performance to avoid the obstacle. The teleoperation performance of the system is validated by the obstacle avoidance experiment of the robotic rollator. The innovation of this paper is as below: • A novel muscle teleoperation system is proposed to combine the benefits of haptic feedback with EMG stiffness estimation .
• A bilateral shared control method is applied to enhance the obstacle performance of the robotic rolator .

II. METHODOLOGY
In order to comply with the shared control method on the robotic rollator, the dynamic model and teleoperation control of the robotic rollator are proposed, and the EMG stiffness during robotic manipulation is estimated. Afterward, the shared control method is introduced. Finally, the force feedback method is presented to solve the robotic kinematics.

A. MODEL OF ROBOTIC ROLLATOR
The kinematic model of the robotic rollator is shown in Fig. 2. In the kinematic model, (X fr , Y fr ) and (X fe , Y fe ) are the VOLUME 8, 2020 coordinate of the front axis and rear axis, respectively. δ s denotes the steering angle while ϕ denotes the yaw angle. R s and P are the steering radius and circle center, respectively. v fr and v re represent the speed of the front and the rear. The wheel tracks is defined as l w . While road curvature and steering radius are in the same condition, The rear wheel speed is obtained as: v re =Ẋ re cos ϕ +Ẏ re sin ϕ Moreover, the kinematic model of the robotic rollator is obtained as:Ẋ fr sin (ϕ + δ s ) −Ẏ fr cos (ϕ + δ s ) = 0 X re cos ϕ −Ẏ re sin ϕ = 0 From Eq. (2), we can obtain: Thanks to the relationship of the steering, the robotic rollator state function can be obtained as: The yaw angular velocity ω y is defined as: Then, the trajectory control of the robotic rollator is obtained as: where ξ s = [X re , Y re , ϕ] T is the system state and u S = v re , ω y T is the control state. The dynamic model of the robotic rollator is defined as follows: where F x1 , F x2 , F x3 and F x4 represent the wheel forces of right front, left front, right rear and left front respectively in X O direction. I r represents the rotational inertia and ω c is the center yaw velocity. In order to obtain the tire's lateral force, the interpretation of the assumptions is shown as: where ψ tcF and ψ TCB are tire cornering angle, CSF and CSB are the stiffness while β is the slip angle.

B. TELEOPERATION CONTROL
We utilize the Novint Falcon haptic robot device, as the teleoperation manipulator to tele-control the robotic rollator [25], [26]. The manipulator is gravity compensated in advance [27], [28]. And the model of Novint Falcon is defined as follows: where q ∈ R m donates the configuration vector of the device, and m is the degrees of freedom of the teleoperation manipulator and in this case m = 3, M s (q) ∈ R m×m donates the inertia matrix, C s (q,q) ∈ R m represents the Coriolis terms, the τ − c represents the haptic feedback generated in section 3.1, and τ t donates the force of the teleoperator. τ c is obtained automatically by the haptic feedback method proposed in section 2.5. The teleoperation control system utilizes the configuration vector q to obtain control input u t in Eq. 10 and then control the movement of the robotic rollator. While designing the teleoperation interface, the dissimilarity between the workspace of the haptic manipulator and the mobile robot should be taken into account. To overcome this limitation, mapping the configuration q to the velocity of the control system of the mobile robot is one of the effective ways [29]. The control signal of the teleoperation system is as following: where u t ∈ R 2 is the control signals of the teleoperation system, T t ∈ R 3 → R 3×m is a transformation matrix to Cartesian space, Q ∈ R 2 → R 2×3 donates the nonlinear mapping.

C. EMG-GUIDED STIFFNESS ESTIMATION
In this paper, an EMG sensor is utilized to obtain the EMG stiffness m EMG , representing filtered EMG signal, is described as: where m(j) is the jth channel EMG signals captured from MYO armband. n donates the eight channels of the MYO armband.
For the purpose of obtaining accurate EMG signals, we filter the EMG signals through low pass filter, envelope and moving average.
Applying the filtered EMG signals, the muscle activation F(j) is obtained as following: K j is the joint stiffness during the task computed as below: where τ is the joint torque for the joint angle q defined as below: where Jm donates the muscle Jacobian defined as below: An estimation of the arm endpoint stiffness (K e ) of the operator is obtained by applying the kinematic Jacobian J that links changes in Cartesian space with changes in joint space, as following:

D. SHARED CONTROL
While the robotic rollator is moving to the target position controlled by the teleoperator, the obstacle is inevitably encountered. So when the robotic rollator approaches an obstacle, the robotic rollator is controlled remotely by to avoid the obstacle. At this stage, human control intentions are directly reflected by EMG stiffness. When approaching an obstacle, a composition force (an attractive force and a repulsive force generated by the shared control mode is received by the robotic rollator to make the robotic rollator away from the obstacle. In the meantime, the haptic device generates force feedback (Eq. (26)) to the teleoperator. Consequently, the force feedback can take the robotic rollator away from the obstacle.  Inspired by [30], a muscle teleoperation system to estimated the EMG stiffness is developed. For Eqs. (16), while the force feedback is received by the human partner from the haptic robot device, the human partner can modify the manipulation in order to avoid the obstacle, and the EMG stiffness will vary accordingly. The attractive force and the repulsive force can be changed by EMG stiffness in the proposed shared control. Specifically, while the robotic rollator approaches an obstacle, the EMG stiffness changes proportionally to increase the composition force in order to accomplish a fast obstacle avoidance.
where S 0 max ≥ S emg ≥ S 0 min is a proportionality parameter to represent the EMG stiffness influence. K max ≥ K e ≥ K min represents the EMG stiffness [31]. S 0 is the scale coefficient of the factor to adjust the EMG stiffness defined as following:

III. SYSTEM DESCRIPTION
Afterward, a shared control interface combining EMG stiffness and APF for the robotic rollator is applied in Fig. 5.
In this interface, the resultant force in the force domain determines the motion of the robotic rollator. An attractive force and a repulsive force make up the resultant force. The robotic rollator is propelled away by the repulsive force, while the attractive force moves the robotic rollator forward to the target point. The combination of APF and shared control Q com is represented as with where Q att is the APF function. Q rep donates the repulsive potential field function. µ att donates the artificial gain parameter. f (p, p ta ) is the distance from the target to the robotic rollator, where p ta is the target's position.
where µ rep donates the repulsion gain parameter. f o donates the influenced radius for the obstacle. Accordingly, the attractive force F att is presented as following The repulsive force F rep is defined as below K vt and K wt represent proportional gains, F is the magnitude of the attractive and potential repulse force, θ F and θ 0 donate the angle of the magnitude force F and the robot's actual instantaneous direction to the x-axis in a global coordinate system. Therefore, the control velocity is defined as: Afterward, the control signal of shared control can be obtained as

A. FORCE FEEDBACK
While the robotic rollator is moving to the goal's position, the distance is d between the robotic rollator and the obstacle. While the safe distance d s is greater than the distance d, force feedback can be generated to the teleoperator through the haptic robot device, as shown in Fig. 7. The control commands can be changed by the human operator applying the muscle teleoperation system. (26) where K M is a positive-definite damping matrix applied to keep the haptic device stable, and K f is a positive gain coefficient for the robotic rollator. d m and d sa are the maximum values of warning distance and the safety distance, respectively. The force feedback of the robotic rollator is large, when the distance is smaller. And the teleoperator can react according to the feedback force to avoid obstacles in advance.
In this paper, the basic controller of the robotic rollator and haptic device is a neural approximation based model predictive tracking controller. The EMG stiffness estimated by EMG S emg > 0 changes the parameters of NPMC controller accordingly. The lateral stability of the control system for the robotic rollator is proposed in [18], hence the control stability can be guaranteed [32].
The control architecture of the robotic rollator is presented in Fig. 6. The robotic rollator is controlled by the teleoperator applying haptic device. And the teleoperator can react according to the haptic feedback of the robotic rollator. The VOLUME 8, 2020 motion intention of the teleoperator is recognized by the EMG sensors wearing in the forearm. Then the shared control is utilized in the robotic rollator. Fig. 6 represents the control architecture of the system. On the master side, the teleoperator wears MYO armband in the forearm and operates haptic devices to remotely control the robotic rollator. The MYO armband is applied to obtain EMG signals to reflect EMG stiffness. Positions and velocities commands in the Cartesian workspace to the robotic rollator are sent by the haptic device.
The robotic rollator is controlled remotely applying TPC/IP protocol. And distance of the obstacle is transmitted back by laser radar. A muscle teleoperation system applying shared control is presented for the robotic rollator to accomplish obstacle avoidance. And the motion intentions of the teleoperator is recognized to control the robotic rollator remotely. The muscle teleoperation system is presented in detail in the following.
In line with the proposed algorithm, EMG stiffness estimation, haptic control and feedback are done with communicating multi-computers, as shown in Figure 8. ROS with User Datagram Protocol (UDP) is applied to communicate between computers. Applying timestamps at the ROS messages ensures the synchronous of the transferring data. The first computer collects the EMG data from the Myo Armband through the blue-tooth protocol, and the second computer obtains positon-velocity control command from the haptic device and provides the teleoperator the haptic feedback from a robotic rollator to give the teleoperator an intuitive feeling. And the computer server communicates wirelessly with the robotic rollator through Transmission Control Protocol/Internet Protocol (TCP/IP).
Finally, the EMG stiffness estimation and the positionvelocity control command are transmitted to the computer server to control the robotic rollator react according to the human intention.

IV. EXPERIMENTS AND RESULTS
In the experiment, the performance and robustness of the muscle teleoperation system are demonstrated.

A. EXPERIMENT SETUP
The teleoperation interface is established to validate the effectiveness of the muscle teleoperation system. Novint Falcon is utilized as the master control device with haptic feedback from the distance of the obstacle. An MYO armband is applied to obtain the EMG signals as the human intention to control the robotic rollator. As the distance of the obstacle increases, the force of the teleoperator needs more to move in that direction. Distance is detected by a lidar that is mounted on the robotic rollator. Fig. 9 shows the experimental environment. There is one obstacle whose position is (2m, 2m), while the target position is (4m, 4m). The teleoperator controls the robotic rollator to get the target position during the obstacle avoidance.

B. OBSTACLE AVOIDANCEE EXPERIMENT
The actual trajectory of the shared control based on and not based on EMG is shown in Fig. 10. And the dangerous zone of the obstacle which is 0.4m far away from the obstacle is labeled in the figure. The start point is the coordinate origin point while the target point is (4,4). In the experiment, e1 represents the first experiment not applying EMG while e2 means the second experiment using EMG with shared control.
The feedback force and composition force are shown in Figs. 11(a) and 11(b). And Fc-e2 is the force applying shared control with EMG stiffness while Fc-e1 is the force utilizing APF control without EMG-based component. From Fig. 11(a), it can easily get the conclusion that a small composition force is imposed in the robotic rollator while only APF control is used. Furthermore, with the EMG stiffness utilizing for shared control, the composition force is much greater, which can enhance obstacle avoidance ability. And the haptic feedback force can help the teleoperator's reaction of obstacle avoidance particularly when the robotic rollator passes the obstacle, which is shown in Fig. 11(b). The red circle represents the required safety distance (0.4m) circle of the obstacle.
The EMG stiffness is shown in Fig. 11(c). The EMG stiffness varies suddenly at 23s while the robotic rollator is approaching the obstacle. In the case, it can be seen that the EMG stiffness varies during the obstacle avoidance, and the performance of obstacle avoidance is enhanced by the components based on EMG.
The velocity performance of obstacle avoidance is shown in Fig. 11(d). During the obstacle avoidance process, the velocity of the EMG-based component (blue curve) is kept at a higher level compared to the component without EMG (red curve).
The displacement and time of the obstacle experiment is shown in table 1. We can find that the completion time applying and not applying EMG stiffness control is 57.66s and 64.22s, respectively. And the total displacement applying EMG stiffness is shorter than not applying EMG stiffness. Maximum warning distance represents the minimum safe distance between the robotic wheel and obstacles. The minimum safety distance of the robot wheel and the obstacle is indicated by the maximum warning distance. Note that the maximum value of the warning distance is an absolute value, regardless of the coordinate point. Because the maximum value of the warning distance varies during the obstacle avoidance, and the average value is applied to represent the minimum safety distance.
It is shown in Table 1, the completion time with EMG stiffness and without EMG stiffness are 64.22s and 57.66s, respectively. In addition, the total displacement in the case with EMG stiffness is shorter than that without EMG stiffness. The maximum warning distance indicates the minimum safe distance between the robotic rollator and the obstacle. It is noted that the maximum warning distance is an absolute value regardless of the coordinate system. Because the maximum warning distance has changed during obstacle avoidance, the average value is utilized to represent the minimum safe distance.
Moreover, from Table 1, it shows that the minimum safety distance applying EMG stiffness is larger than the minimum safety distance without applying EMG stiffness. We can get the conclusion that the proposed muscle teleoperation system based on EMG stiffness is able to obtain a larger minimum safety distance.

V. CONCLUSION
In this paper, a muscle teleoperation interface applying the shared control method to avoid obstacles is developed. Haptic feedback and EMG stiffness estimation are integrated to provide a large composition force. Then the composition force is implemented with the APF method to obtain a larger composition force to keep the robotic rollator away from the obstacle in advance. This algorithm is superior to the traditional APF algorithm regardless of the required time or trajectory length. The effectiveness of the muscle teleoperation interface is demonstrated by the experimental results.
Compared with the traditional APF method applying in obstacle avoidance, the feedback force and composition force are more responsive, as the influence of human intention can be effectively accounted by the robotic rollator. It should be emphasized that with adding an EMG-based controller, the robotic rollator can vary attractive and repulsive forces based on EMG stiffness, and corresponding force feedback is generated to the teleoperator to achieve feedback control to the robotic rollator. According to the experimental demonstration, the muscle teleoperation system can implement the predictability of avoiding obstacles and force feedback to the teleoperator is provided in order to control the robotic rollator in his intention. In this article, EMG signals are utilized to give the feedback signal of the obstacle to the teleoperator to warn the existence and distance of the obstacle. The EMG stiffness is transferred as a control parameter in the shared control system to increase the composition force proportionally while the robotic rollator is approaching an obstacle. Moreover, the force feedback can make it more difficult for the robot rollator to move towards obstacles during teleoperation. Meanwhile,the teleoperator is able to control the robot according to the force feedback from the haptic robotic device.
The EMG stiffness is transferred to a proportionality coefficient to increase the composition force of the shared control when the robotic rollator moves toward an obstacle. In addition, the force feedback can make the robotic rollator move toward the obstacle more difficult in the teleoperation. Meanwhile, the teleoperator is able to control the robot with the force feedback from the haptic device.
In this scenario, the muscle teleoperation system can reduce the workload of teleoperators. The muscle teleoperation system is able to extend to autonomous delivery robots, rescue robots, and medical robots. In the presence of an obstacle, the minimum distance between the robotic rollator and obstacle can be calculated by the muscle teleoperation system. Then the minimum distance and the composition force are combined to achieve obstacle avoidance.
The obstacle avoidance can be achieved regardless of the skill of the teleoperator. Hence with this muscle teleoperation system, the training time of the teleoperation can be reduced while the safety is ensured. EMG is utilized to add human intention in the control system and to reduce the dependence on operational skills. In future work, multiple control methods such as bilateral control will be considered to use in the robotic rollator. And with the distance increases, the time delay of the control should be considered as another challenge. In this scenario, the topic is focused on using EMG signals to achieve obstacle avoidance performance. And using EMG signals to monitor the physical condition of the patient in a robotic rollator can be another future work. LONGBIN ZHANG received the M.Sc. degree in control engineering from the South China University of Technology, Guangzhou, China, in 2017. She is currently pursuing the Ph.D. degree in robotic exoskeletons for patients with motor disorder with the Department of Mechanics, KTH Royal Institute of Technology, Stockholm, Sweden. She worked on bio-signal processing and exoskeleton control. She has participated in some research projects sponsored by the Promobilia Foundation and the Swedish Scientific Council, Sweden. Her main research interests include simulation of human movement, robotic exoskeleton assistive strategies, EMG control, and optimizing and reinforcement learning control. She served as a Reviewer for over ten scientific journals, such as the IEEE TRANSACTIONS ON