Deep Neural Network Approach in Human-Like Redundancy Optimization for Anthropomorphic Manipulators

Human-like behavior has emerged in the robotics area for improving the quality of Human-Robot Interaction (HRI). For the human-like behavior imitation, the kinematic mapping between a human arm and robot manipulator is one of the popular solutions. To fulfill this requirement, a reconstruction method called swivel motion was adopted to achieve human-like imitation. This approach aims at modeling the regression relationship between robot pose and swivel motion angle. Then it reaches the human-like swivel motion using its redundant degrees of the manipulator. This characteristic holds for most of the redundant anthropomorphic robots. Although artificial neural network (ANN) based approaches show moderate robustness, the predictive performance is limited. In this paper, we propose a novel deep convolutional neural network (DCNN) structure for reconstruction enhancement and reducing online prediction time. Finally, we utilized the trained DCNN model for managing redundancy control a 7 DoFs anthropomorphic robot arm (LWR4+, KUKA, Germany) for validation. A demonstration is presented to show the human-like behavior on the anthropomorphic manipulator. The proposed approach can also be applied to control other anthropomorphic robot manipulators in industry area or biomedical engineering.


I. INTRODUCTION
H MAN-LIKE has attracted increasing research interest in the past decades in various areas, such as industrial application, service robot, and medical sector, etc.. In particular for the cases that robots and humans share the workspace [1] [2], for example, close collaboration between human operators and industrial robots manufacturing, assistance system for elderly users, etc., Human-Robot Interaction(HRI) plays a vital role in these practical applications [3] [4]. It has been proven that both humanoid appearances and humanlike motion behavior can facilitate task performance of HRI [5]. For the anthropomorphic serial manipulators [5], for example, LWR4+ (KUKA, Augsburg, Germany), Justin robot (Institute of Robotics and Mechatronics, Wessling, Germany) and YuMi (ABB, Zurich, Switzerland), although with a similar mechanical structure with human arm, a human-like behavior of the robot arm pose can be an enhancement of this topic because it provides a more social and reasonable movement in HRI [6].
Several studies had been performed to introduce humanlike behavior for improving the performance of human-robot collaborative tasks. Beretta et al. [7] achieved an adaptive human-like hands-on control to reach and target tasks in surgery with a redundant robot. A reaching task with the human-like motion for robot-environment interactions had been studied in [8]. Huang et al. [9] transferred humanlike impedance to control a dual-arm Exoskeleton Robot. De Momi et al. [10] used the feed-forward Artificial Neural Network (ANN) algorithm to obtain human-like motion for trajectory planning. However, the above works focused on the human-like motion only on the end-effector of the robot without consideration of the arm pose, which is the main obstacle to whole-body motion control.
Instead of human-like motion on the end-effector, the robot arm pose can achieve human-like behavior by utilizing the kinematic redundancy [11]. The definition of elbow elevation angle for a human arm, shown in Figure 1, was adopted by Kim et al. [12] for realizing human-like arm motion generation using captured data. Zanchettin et al. [13] proposed to resemble a human-like behavior at a kinematic level, in order to avoid any unease or discomfort (like fear or shock) to the nearby humans. They redefined the elbow elevation angle, ψ, shown in Figure 2, for a human arm mapped on the robot as a swivel motion. Compared to the biomimetic approach using inverse kinematics to achieve human-like kinematic behavior developed in [14], the introduced swivel motion notion provides a more general strategy regardless of the robot kinematic structure. Except for controlling the hand pose of the redundant manipulator, resolving the human-like swivel motion using its redundancy is applicable for most typical industrial serial manipulators. [15] Ajoudani et al. [16] achieved natural redundancy resolution on a dual-arm manipulator. A wrist-elbow-in-line method was introduced by Liu et al. [17] to map the elbow angle from human demonstrations on the real robot to obtain a human-like kinematics solution. Yang et al. [18] utilized teaching by demonstration scheme integrated the tutor's motor functionalities into the robot's control architecture, transferring the motion behavior from the human to robot. Furthermore, the relation between swivel angle [19] and hand pose had been analyzed in [20]. A human-like behavior was achieved with the nonlinear regression relationship [21], [22] between the swivel angle and the hand pose [23]. The above research proved the human-like behavior was achievable if an accurate nonlinear regression model [24], [25] with multi-inputs could be obtained. The polynomial regression method [26] had been used to describe nonlinear phenomena such as the progression of disease epidemics FIGURE 2: Achieving human-like behavior on the KUKA anthropomorphic manipulator. S, E, and W are the coordination position of the shoulder, the elbow, and the wrist, separately. The human-like defined elbow angle ψ is implemented with the swivel motion of the robot arm between the vertical reference plane (blue) and the arm plane (red). and distribution of carbon isotopes in lake sediments [27]. However, this method needed prior knowledge of the inputs for selecting the best parameters. Therefore, semi-parametric [28] and non-parametric [29], [30] regression approaches were presented for predicting the vectors semi-with or without related a predetermined form but was constructed according to information derived from the input sequences. However, they required larger sample sizes to increase regression accuracy. Recently, ANN approaches had been applied to nonlinear regression analysis for geophysical explorations [31], biomedical applications [32], and motion tracking [33]. These studies proved that ANN-based regression algorithms ensured high accuracy by comparing to traditional nonlinear methods [34] [35]. In our previous work [10], we utilized a single layer feed-forward neural network (FFNN) algorithm to build the nonlinear regression model for achieving humanlike behavior to do the tasks in real-time. It was validated in the teleoperated surgery scenario, and the studies show that people want a robot with human-like behavior in the shared room. However, with the growing inputs dimensions and interference components, the previous shallow layer ANN regression methods cannot fit these new situations. While the Deep Neural Networks (DNN) is capable of meeting the requirements such as better accuracy, noise robustness, and decreased computational time.
This paper presents a novel nonlinear regression algorithm to map the relation between swivel angle and the hand pose using deep (convolution) neural network (DNN) approach to improve the ability of nonlinear regression analysis of human-like motion model. The DNN model aims to map the time-varying multi-inputs (the 6-D task pose), including Cartesian positions x,y,z and Euler angles θ x , θ y , θ z and the output elbow swivel angle ψ. At the same time, accuracy, noise robustness, and computation efficiency of the regression model should be respected for real-time motion control. The input of the model is the target pose, and the output is the swivel motion angle. The vision with skeleton tracking is used to collect human motion data. This work aims to map the human swivel angle on the robot arm during the tracking tasks. Finally, the effectiveness of DNN is validated using the human motion data, and we translated the DNN model to manage redundancy control a 7 DoFs anthropomorphic robot arm (LWR4+, KUKA, Germany). As a demonstration, the anthropomorphic manipulator control using the DNN model is presented with human-like behavior.
The paper is organized as follows: the corresponding methodologies and the system architecture are presented in section II and III, separately. Section IV compares the regression performance between DNN and other ANN methods on the sample dataset. Moreover, it demonstrates the experimental validation and results of the proposed methodology evaluated with the KUKA LWR4+ robot. Finally, section V draws a conclusion and delineates avenues for further work.

II. METHODOLOGY
In order to implement the human-like behavior on the humanoid robot, the kinematic model of the human arm is presented and the swivel motion during the manipulation task is analyzed. After that, a data acquisition system with skeleton tracking and DNN-based model training algorithm are introduced. Finally, the built DNN model is transferred to the analytical kinematic solution of the robot.

A. KINEMATIC MODELING
To achieve human-like behavior on the KUKA anthropomorphic manipulator (Figure 2), the elbow swivel angle ψ is already defined in our previous work [10], shown in Figure 3. Also, the human arm is simplified as a rigid kinematic chain connected by three basic joints (shoulder, elbow and wrist) including 7 DoFs, shown in Figure 4. By establishing the table of Denavit-Hartenberg (D-H) parameters [36], defined in Table 1, it is feasible to calculate the hand pose 0 T 7 using forward kinematics function [37] through the joints coordinates (θ j , j = 1, 2, · · · 7) as follows: The joint angles (θ i , i = 1, 2, · · · , 7) can be obtained based on the geometry relation. Then, the swivel angle ψ will be calculated according to the vector relation between the reference plane and arm plane [10].

B. HUMAN MOTION DATASET ACQUISITION AND PREPROCESSING
After the kinematic modeling, the data acquisition system using KINECT V2 (Microsoft, USA) for skeleton tracking and swivel motion calculation is developed [38]. The acquisition software of the human arm motion is implemented on MATLAB 2018b. As it is shown in Figure 5, the subject is working in the workspace within the range of vision system which is overlaid by the skeleton viewer. When the "Start" button is pressed, the real-time vision system could acquire the joint position coordinates and calculate the joints angles through the geometry relation explained in the human arm model which is capable of representing the same human motion in the video. Meanwhile, it can compute the hand target pose based on Eq. 1. Finally, the saved dataset with 6-D inputs (hand pose) and output (swivel angle) are adopted to train the DNN model for further regression analysis. It should be noticed that the motion-capture experiments of the human arm were performed without considering the robot kinematic parameters. The motion data was collected only considering the right arm of the subject. To resolve these issues, we proposed a deep convolutional neural networks (DCNN) structure for reconstruction accuracy enhancement, fast computation and noise robustness. The convolutional network and ReLU function are widely known to be less computationally expensive and robust compared to the rest ANN-based algorithms [39]. Figure 6 shows the details of the DCNN architecture. The inputs are the 6-D task pose of human arm model, namely X = [x, y, z, θ x , θ y , θ z ]. Where x,y, and z are the Cartesian positions and θ x , θ y , and θ z are Euler angles. To improve the performance of the DCNN model, the inputs are transformed as a 6 × 3 matrix by whereX denotes the average of X and σ(X) is the standard deviation of X.
Parameter vector set Θ accounts for all the parameters of DCNN model, including all of the weights ω and bias vectors b in each layer. A nonlinear regression problem aims to find the optimal parameters set Θ by computing the minimum least squares between the predicted valueψ t and the real value φ t as follows: After acquiring the DCNN model with optimal parameters, it needs to validate its performance in the testing procedure with a batch learning mechanism by comparing the common regression indexes. The Mean Absolute Error (MAE), Mean Square Error (MSE), and Pearson correlation coefficient (ρ) are widely used for evaluating the results of regression analysis [42] and judging the similarity between predicted and real curves [43] defined as follows:   Where t is the number of observations,ψ andψ are the average of the predicted valuesψ and ψ, while σψ andψ are the standard deviation ofψ and ψ. The best score for correlation coefficient ρ is 1 and for other measures is 0.

D. ROBOTIC HUMAN-LIKE REDUNDANCY RESOLUTION
To transfer the DCNN model to the robot, a kinematic redundancy optimization solution with the consideration of joint limitations has been proposed in our previous work [10]. Its feasibility has been validated on the real KUKA robot with telemanipulation tasks. The swivel angle ψ is defined by the angle between the reference plane and the actual arm plane. To control the end-effector for tracking task, an interpolation method is introduced so to move towards the final pose X f ∈ R 6 smoothly as [44]: where k > 0 is a positive coefficient. X d ∈ R 6 is the desired target pose. The target pose of the robot tool and the humanlike swivel angle are set as the input and output of the DCNN model. Then, it generates an interpolation strategy based swivel motion trajectory [10]. To reach the desired target pose X d and the desired swivel motion angle ψ d , the velocitybased control can be introduced. The main task projection relation between the end-effector's velocityẊ and the joint velocityq can be expressed by: where J (q) ∈ R 6×7 is the Jacobian matrix from the endeffector to the robot base, which represents a mapping relation between task-space and robot joint velocitiesq. Nullspace projection is a general solution to resolve the redundancy of a redundant robot, which can be found as follows: where J E ∈ R 3×4 is the Jacobian matrix from the elbow of the robot to the robot base, which represents a mapping relation between swivel motion angle and joint velocities. And u ψ ∈ R 3×1 is the velocity director of swivel motion, defined as: where − → SE is the vector from the shoulder to the elbow of the robot and − − → EW is the vector from the elbow to the wrist of VOLUME 4, 2016 the robot, as it is shown in Figure 2. In this paper, we assume that the robot is always far away from the singularity, and its pseudo-inverse matrix J + exists. The corresponding redundancy resolution kinematic controller is shown in Figure 8.  Figure 7 is the developed system architecture including two parts as follows:

III. SYSTEM DEVELOPMENT AND ARCHITECTURE
(1) human motion analysis. After filtering the acquired data from the KINECT V2 equipment and calculating the joints angle, the human arm kinematic model could derive the target pose. Finally, the collected hand pose and its corresponding swivel angle (with 30Hz sample frequency) are utilized to build the DCNN model. (2) human-like redundancy resolution control. The established DCNN model could achieve the smooth joints configuration for real-time tracking tasks according to the developed theory of inverse kinematic solution in our previous works [10].

IV. EXPERIMENT AND DEMONSTRATION
To evaluate the effectiveness of DCNN model for human-like redundancy optimization, comparisons are performed with ANN-based and recurrent neural network (RNN) models, such as long-short-term-memory (LSTM). The optimization parameters of LSTM structure are set as follows. The initial learning rate is 0.05 with 0.05 drop factor and 100 drop period. The minimum batch size is 50. We select adaptive moment estimation(adam) optimizer as the replacement optimization algorithm. The acquisition software of the human arm motion is implemented on MATLAB 2018b running on Windows PC with 16.0GB RAM and 2.80GHz Intel core. The final demonstration experiment using KUKA robot is conducted with the saved DCNN model to validate its feasibility.

A. PERFORMANCE COMPARISONS OF DCNN MODELING
Since the human limb length varies, the D-H parameters is different between the subjects. Moreover, this paper aims to validate the efficiency for training the human-like motion model. Hence, we ignore the difference of limb length and hire one subject for the data acquisition by performing natural reaching motions in the specified cubic task space (surgeons' hand workspace). The data acquisition is with written informed consent from the subject in accordance with the Declaration of Helsinki. The subject is commanded to stand in front of the table and to perform the hand motion in a cube workspace (0.21 × 0.297 × 0.18m 3 ). The workspace is in front of the subject with a distance around 0.2m, and it is 0.69 m higher than the ground 0.69m. The detailed geometry information of the motion data set and experimental protocol of the data collection has been described in our previous work [10]. The acquired dataset are divided into the training set (24 trajectories) for building the DCNN model which is comparable with the subject's motion variability on the swivel trajectory (7700 samples), and the two testing datasets (2 arbitrary movements trajectories) for evaluating the regression performance of DCNN model (each have 1000 samples). Figure 9 shows  The proposed DCNN model could also obtain a faster regression speed than LSTM and ANN-based model. Table  2 displays the comparative average and sum of testing time on the two trajectory tasks. The results prove that the DCNN model is the fastest approach for predicting an output than the other methods. It only needs 0.0038 seconds to output a result and 3.83 seconds to predict all of the 1000 results. Figure 10 shows the prediction values of DCNN model on the training and two testing datasets. The top picture describes the predicted and observed curves (7700 samples) in the training process with a lower predictive error, while the last two pictures display the same curves working on the two testing datasets. The DCNN model is proved as a noise robustness approach. In a dynamic environment, various noises decrease regression accuracy. For the experiment, we add 10dB and 40 dB into the two testing datasets. Table 3 shows the comparison errors (MAE and MSE) on both trajectory datasets. The proposed DCNN structure obtains the lowest errors than the other methods which are labeled by bold numbers.

B. DEMONSTRATION OF HUMAN-LIKE REDUNDANCY OPTIMIZATION USING DCNN MODEL
After the validation of the performance of DCNN in the human-like model training, we demonstrate the human-like redundancy optimization using the built DNN model on the KUKA robot. The robot pose is calculated based on an interpolation algorithm and it is used as the input of DCNN model. Then the trained model predicts the humanlike swivel angle. The developed inverse kinematic mapping strategy in our previous works [10] is introduced to get the joints solution. As it is shown in Figure 11, with the trained model, the robotic arm can replicate human kinematics strategies for performing tracking task, achieving human-like arm posture during the manipulation of a task.

V. CONCLUSION AND FUTURE WORK
This paper proposed a novel deep convolutional neural network structure (DCNN) in human-like redundancy optimization for anthropomorphic manipulators. It features with reconstruction accuracy enhancement, fast computation and noise robustness. Finally, we validated the human-like re- dundancy optimization control of a 7 DoFs anthropomorphic robot arm (LWR4+, KUKA, Germany) using the trained DCNN model. As a demonstration, the anthropomorphic manipulator control using the DCNN model is presented with human-like behavior. The experiments prove that DCNN model not only could acquire a higher reconstruction accuracy than ANN-based and LSTM models, but also cost less time to predict the results. Meanwhile, the DCNN model has powerful noise robustness than the other methods (shown in  table 3). Although it can get a better model by adjusting the parameters of ANN-based and LSTM models, the proposed DCNN model shows promising noise robustness against a wide range of noise even. The joints position and speed limitation have also been taken into account in the humanlike redundancy optimization. The experiments have been conducted on LWR4+ simulator to verify the performance of the proposed algorithm. Results show that the robot can achieve human-like behavior and is feasible for control. The online redundancy method is generalizable, can be applied in a wide variety of anthropomorphic robot arms with similar Kinematic structure.
However, further works will involve more subjects and more trajectories with different geometries for the general model validation. Also the improved human-robot impedance control [49] will be utilized to enhance the safety of human-robot interaction instead of rigid kinematic control.