Natural Walking Trajectory Generator for Humanoid Robot Based on Three-Mass LIPFM

In this paper, a dynamic model named Three-mass Linear Inverted Pendulum plus Flywheel Model (TLIPFM) is proposed to approximate the walking motion of the humanoid robot. The mass distribution of the robot and the angular momentum of the robot are simultaneously considered to construct the dynamic model. In the mass distribution of the robot, the overall robot is divided into three parts: 1) the whole upper body of the robot, 2) the thigh of the robot’s support leg, and 3) the calf of the robot’s support leg, and these three weights are considered to be three masses in the proposed TLIPFM. In the angular momentum of the robot, a flywheel joint is configured at the Center of Mass (CoM) of the robot to obtain the rotational torque of the robot in biped walking. In addition, a natural walking trajectory generator based on the TLIPFM and a moving Zero Moment Point (ZMP) reference is derived to generate ZMP trajectory, CoM trajectory, and foot trajectory in three-dimensional space to let the humanoid robot walk stably. In the experiment, some results are presented to illustrate that the proposed TLIPFM can effectively reduce the model error and the proposed TLIPFM-based walking trajectory generator makes the walking trajectories more natural and walking more stable. Moreover, the proposed method is implemented on a real small-sized humanoid robot to illustrate its feasibility and practicability for the real-time biped walking.


I. INTRODUCTION
The research and application of robots has been widely explored and discussed. The spherical robot [1], [2] and the wheeled robot [3], [4] are designed to facilitate the easy movement, and they are intrinsically stable robots, so they can be easily prevented from tipping over. However, these robots cannot move in some complex environments, such as stairs or uneven terrain. The humanoid robots [5]- [8] are designed to imitate human behavior to move and step in the terrain, and they are intrinsically unstable robots, so they are more easily to fall over. Especially, only two legs are equipped on the humanoid robot, and a single leg supports the ground while the robot moves to perform biped walking. Hence, the biped walking is one of the significant technical topics in the research field of humanoid robots.
The associate editor coordinating the review of this manuscript and approving it for publication was Gang Mei .
In the discussion of the biped walking of the humanoid robot, several concepts have been proposed, such as central pattern generators (CPGs), simplified coupled oscillators, zero moment point (ZMP), and linear inverted pendulum model (LIPM). The CPGs are utilized to affect in the movement of biological rhythms and spontaneous steady oscillation signals are generated by the mutual inhibition networks and constant activation [9]- [11]. The simplified coupled oscillators can be divided into a balance oscillator and two movement oscillators [12]- [14]. The center of mass (CoM) trajectory and both foot trajectories can be designed through one balance oscillator and two movement oscillators, respectively. The ZMP equation is used to describe the link characteristics of the humanoid robot, such as the link mass and the link height, and the walking trajectory is generated by using preview control [15]- [17]. The LIPM is viewed as a single-mass model to be concentrated at the CoM and move along the horizontally defined plane based on mathematical methods of dynamics [18]- [22]. The LIPM equation of VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ motion is built to design the CoM trajectory, which is the most important function of human walking. The stable biped walking is expected to implement for the humanoid robot through aforementioned approaches. In recent years, the LIPM concept is extended to increase the stability of the humanoid robot. For example, a linear inverted pendulum plus flywheel model (LIPFM) was proposed and applied to the standing robots [23]- [25]. The rotational inertia, which is the natural human movement, is enabled to control the angular momentum of the humanoid robot. In addition, a double-link linear inverted pendulum model (DLIPM) was proposed and applied to the walking robots [26], [27]. Compared with the LIPM, the DLIPM equation of motion with the mass of the support leg is built to design the CoM trajectory for the humanoid robot. Besides, the relationship between LIPM and ZMP is applied as the walking reference for forward walking. The ZMP feature is adopted to ensure the walking reference inside the support polygon and achieve the stable biped walking for the humanoid robots [28], [29]. Moreover, the natural ZMP trajectory was designed to imitate human walking by the concept of moving ZMP reference [30], [31] or heel-to-toe direction [32], [33]. These approaches are all extended from the LIPM and are expected to be more similar to human behavior or closer to the actual situations.
ZMP is the well-known stability criterion for biped walking of the humanoid robot. The stable biped walking can be validated through the ZMP response which can be measured by force sensors placed on the bottom of each sole [34]. The force sensors are mounted separately at fixed positions of the sole to detect its force and calculate the ZMP response. When performing biped walking, the ZMP response can be recorded at the same time to measure the walking performance of the humanoid robot [35], [36]. Especially, the ZMP criterion must be obeyed to ensure the stable biped walking, that is the ZMP response must be within the maximum and minimum boundaries [37], [38]. The humanoid robot may fall if the ZMP response exceeds the maximum and minimum boundaries.
In this paper, a Three-mass Linear Inverted Pendulum plus Flywheel Model (TLIPFM) is proposed to approximate the walking motion of the humanoid robot. The TLIPFM is based on the concept of LIPM and adding some practical conditions, such as the structure of the mass distribution of humanoid robots and the angular momentum of the human walking behavior. Therefore, the proposed TLIPFM is a more realistic model. In addition, a natural walking trajectory generator based on the proposed TLIPFM is generated for the humanoid robot to implement stable biped walking. First, a natural ZMP trajectory is designed by moving ZMP reference to imitate human walking. Then a CoM trajectory is derived based on the proposed TLIPFM and the natural ZMP trajectory. Finally, a foot trajectory is derived and used to match the CoM trajectory to achieve stable biped walking. The experimental results show that the proposed method can make the robot walk more stably.
The rest of this paper is organized as follows. In Section II, the proposed TLIPFM is described and its equations of motion are derived. In Section III, the natural walking trajectory generator based on the proposed TLIPFM is presented for the humanoid robot to achieve biped walking. The natural ZMP trajectory, CoM trajectory, and foot trajectory are generated to compose the natural walking trajectory generator for forward walking. In Section IV, a small-sized humanoid robot is used to execute practical experiments. Some experimental results are presented to validate the feasibility and practicability of the proposed method. Finally, this paper is summarized in Section V.

II. DYNAMIC MODEL OF HUMANOID ROBOT
The humanoid robot is a complex system structure with multiple links and high dimensions. Hence, in the realization of real-time biped walking for humanoid robots, a simplified dynamics model is needed, which can approximate the walking motion of a humanoid robot and reduce the consumption of computing resources. The Linear Inverted Pendulum Model (LIPM) is a simplified dynamic model often used in real-time biped walking, because LIPM can be easily implemented in real robots with relatively low computational cost. There are three basic assumptions in LIPM, but one of them is that the weight of the whole robot is considered to be a single mass and is concentrated at the CoM. The mass of the support leg is assumed to be massless, which is far from the actual situation. Therefore, the mass of the support leg is considered in the model of double-link LIPM (DLIPM) [26] to eliminate the conficting assumption about the LIPM. There are four assumptions in DLIPM, but the weight of the support leg is assumed to be one mass. In the proposed Three-mass Linear Inverted Pendulum plus Flywheel Model (TLIPFM), the whole robot divided into three parts: 1) the whole upper body of the robot, 2) the thigh of the robot's support leg, and 3) the calf of the robot's support leg. The weight of the whole upper body of the robot is considered to be a single mass, and the weight of the thigh and calf of the robot are respectively also considered to be a single mass and concentrated in the middle of its link. In addition, the angular momentum of the humanoid robot during walking will also affect its stable biped walking. Therefore, a flywheel joint is configured at the CoM of the robot so that the angular momentum of the human walking behavior is also considered in the proposed TLIPFM.
Based on Cartesian coordinate system, the walking motion of the humanoid robot can be simplified by the proposed TLIPFM. A description of forward walking in the sagittal plane is shown in Fig. 1 and six assumptions of the proposed TLIPFM can be described as follows: 1) The weight of the whole upper body of the robot is considered to be a single mass M u and is concentrated a point (x, y, z c + l u ), which is the middle of the upper body.
2) The weight of the thigh of the support leg is considered to be a single mass m t and is concentrated a point  from the CoM is always perpendicular to the horizontal plane. 6) The flywheel joint of the robot is configured at a point (x, y, z c ), which is the CoM of the robot. Since the mass distribution and the angular momentum of the robot are simultaneously considered in the proposed TLIPFM, compared with the models of LIPM and DLIPM, the proposed dynamic model will be closer to the actual walking motion of human.
The angular momentum is also an important effector in the stable biped walking of the humanoid robot so that the flywheel joint is considered in the TLIPFM to approximate it. The rotational torque τ of the angular momentum in the flywheel joint can be described by where J is the rotational inertia, φ andφ are the rotation angle and angular acceleration of the CoM, respectively. In the sagittal plane of the single support phase, the sagittal TLIPFM equation of motion can be derived by the torque balance and described bÿ where x is the position of the CoM in the sagittal direction, z c is the constant height of the CoM, M u is the mass of the whole upper body, m t and m c are respectively the mass of the thigh and calf of the support leg, g is the gravity constant, and l u is half the length of the upper body.
Similarly, in the lateral plane of the single support phase, the lateral TLIPFM equation of motion can be described bÿ where y is the position of the CoM in the lateral direction. Hence, in the proposed TLIPFM, the state space equation of motion of the CoM in the sagittal plane and the lateral plane can be described by is the rotational torque parameter of the equation of motion and T TF is the natural period of the TLIPFM. Note that (4) is not only related to the gravity constant g and the height z c of the CoM, but also related to the actual mass distribution (M u , m t , m c ) of the robot and the angular momentum τ . The mass distribution and the angular momentum are incorporated into the model that TLIPFM is closer to the actual walking motion of human. Hence, the TLIPFM equation of motion can be adopted for the humanoid robot to design the CoM trajectory and achieve stable biped walking.

III. NATURAL WALKING TRAJECTORY GENERATOR
ZMP is the most well-known stability criterion for biped walking. It is necessary to establish a ZMP trajectory to be the walking reference of the robot for the stable biped walking. In this paper, the proposed TLIPFM is adopted to generate a stable CoM trajectory in three-dimensional space, and then the feet need to match the CoM trajectory planned by TLIPFM through generating foot trajectory in three-dimensional space. In this way, a walking trajectory generator is established for the humanoid robot to achieve stable biped walking. Therefore, in the proposed walking trajectory generator, there are three parts: (A) ZMP trajectory, (B) CoM trajectory, and (C) foot trajectory. They are described as follows: A. ZMP TRAJECTORY During the walking process of the humanoid robot, the walking reference of the humanoid robot based on the Zero Moment Point (ZMP) is adopted to ensure that the robot can walk normal and stable. As shown in Fig. 2, the step location with fixed ZMP reference (ZMP position) p c of the left foot and the right foot is the position of the support foot and represented by where p c x and p c y denote the coordinate of ZMP in the sagittal plane and the lateral plane, respectively. Through the step VOLUME 8, 2020 Step locations with fixed ZMP reference in the horizontal plane.
location of each step of the humanoid robot, the walking reference trajectory in the sagittal plane and lateral plane can be respectively described by and where S L is the step forward distance in the sagittal plane, S W is half the distance between the feet in the lateral plane, n is the n-th step, N is the total number of steps, and T is the period in one step, which is half of the walking period. u(·) is the unit step function. Besides, S first is described by It is used to describe the starting support foot. For example, S first is set −1 to let the left foot first start. Therefore, the step location p c of the walking reference described by (6) and (7) are also decided by the starting support foot S first . Based on the walking reference trajectory, the ZMP position p c can be generated which is the motion behavior of the humanoid robot during the single support phase (SSP). In addition, the double support phase (DSP) and moving ZMP reference can be added that not only extend the support polygon of the support foot but also walk naturally from heelto-toe to improve safety and stability during biped walking, as shown in Fig. 3 and Fig. 4. Therefore, the step period T of each step of the humanoid robot can be described by where T SSP and T DSP are respectively the period in the single support phase and the double support phase. The ZMP trajectory of the walking trajectory generator is generated based on the moving ZMP reference, which is the natural ZMP trajectory [26], [31], so that the ZMP of the  Step locations with moving ZMP reference in the horizontal plane.
humanoid robot during biped walking can be similar to the human ZMP. The natural ZMP trajectory in the sagittal plane and the lateral plane is shown in Fig. 5 and a single step can be respectively described by and where the time between t 1 and t 2 is the period in the single support phase T SSP = {t|t 1 < t ≤ t 2 } while others are the period in the double support phase T DSP = Similarly, based on the lateral step location p c y , the lateral TLIPFM equation of motion described by (3) can be rewritten as follows:  (13) Hence, based on the step location, the state space of the TLIPFM equation of motion in the sagittal plane and lateral plane described by (4) can be rewritten as follows: Through the analyzation of the second-order differential equation, the sagittal CoM trajectory described by (12) can be derived based on the sagittal step location p c x in the single support phase. The position and velocity of the CoM in the sagittal plane can be respectively described as follows: Similarly, the lateral CoM trajectory described by (13) can be derived based on the lateral step location p c y in the single support phase. The position and velocity of the CoM in the lateral plane can be respectively described by where (x(0),ẋ(0)) and (y(0),ẏ(0)) are respectively the initial position and velocity of the CoM in the sagittal plane and lateral plane. Hence, based on the step location, the state space of the CoM motion in the sagittal plane and lateral plane can be respectively described by VOLUME 8, 2020 and In this paper, the double support phase and moving ZMP reference are adopted to generate the natural ZMP trajectory p zmp and replace the ZMP position p c , the step location in the single support phase, for biped walking. Hence, based on the natural ZMP trajectory, the state space of the CoM motion in the sagittal plane and lateral plane described by (19) and (20) can be rewritten as follows: and Applying the differential equation, the sagittal CoM trajectory described by (21) can be derived based on the sagittal natural ZMP trajectory p zmp x described by (10), the position and velocity in the sagittal plane can be respectively described by Similarly, the lateral CoM trajectory described by (22) can be derived based on the lateral natural ZMP trajectory p zmp y described by (11), the position and velocity in the lateral plane can be respectively described by +y 1 (t) + y 2 (t) + y 3 (t) (25) anḋ where Based on the natural ZMP trajectory, the CoM trajectory in the sagittal plane and lateral plane are shown in Fig. 7. When the forward constant angular momentum (τ = 0) and  the forward and backward random angular momentum based on the natural ZMP trajectory are considered in the forward walking, the position and velocity of the CoM trajectory in the sagittal plane and the lateral plane are respectively shown in Fig. 8 and Fig. 9. The dotted line is the natural ZMP trajectory. The solid and dashed lines are the CoM trajectories with the angular momentum and without the angular momentum, respectively. As shown in Fig. 8(a), at the starting position of walking, the velocity of the CoM trajectory in the sagittal plane with angular momentum rises more rapidly than the velocity of the CoM trajectory without the angular momentum, because the robot is affected by continuous and fixed angular momentum. During the walking period of biped walking, the velocity of the CoM trajectory with angular momentum is still faster than the velocity without the angular momentum. At the stopping position of walking, in order to stop at the end position, the velocity of the CoM trajectory with angular momentum drops sharply than the velocity of the CoM trajectory without the angular momentum. On the other hand, the forward and backward random angular momentum is used to simulate that the humanoid robot is affected by the front and back directions in the real walking process. As shown in Fig. 8(b), the velocity of the CoM trajectory with the angular momentum fluctuates more than that of the CoM trajectory without the angular momentum. As shown in Fig. 9(a) and Fig. 9(b), because the CoM trajectory in the lateral plane is not affected by angular momentum, the CoM trajectory with and without the angular momentum are the same. In addition, the position of the CoM trajectory in the horizontal plane is shown in Fig. 10. Note that the position of the CoM trajectory with the angular momentum is forward and fluctuates more than the position of the CoM trajectory without the angular momentum, because the humanoid robot is affected by the angular momentum.

C. FOOT TRAJECTORY
For the humanoid robot can be able to walk, swing foot of the humanoid robot should be lift high and far enough. The swing foot trajectory is designed by using a cycloid function, which is similar to the human ankle circling the tiptoe.   The left foot trajectory is the same as the right foot trajectory except for a period T delay. The foot trajectory in the parasagittal plane is shown in Fig. 11. The swing foot is as follows: , Amp x , Amp y , and Amp z are respectively the step length, the step width, and the step height that defined as (p c x (nT ) − p c x ((n − 2)T ))/2π, W y , H z /2. Based on Cartesian coordinate system, ZMP trajectory, CoM trajectory, and foot trajectory in three-dimensional space generated by the proposed walking trajectory generator are shown in Fig. 12. Through inverse kinematics, these trajectories can be transformed into the desired joint position to let the robot safely walk.

IV. EXPERIMENTAL RESULTS
In this section, a small-sized humanoid robot named TKU-X is used to verify the effectiveness of the proposed method. The TKU-X robot is developed by our laboratory. It has one  head, two arms, one body, and two legs to imitate the human appearance. It has 23 degrees-of-freedom (DOFs). As shown in Fig. 13, the head has 2 DOFs, each arm has 4 DOFs, the waist have 1 DOF, and each leg has 6 DOFs. Its height and weight are respectively 56.45 cm and 4.95 kg. The detailed mass distribution of TKU-X Robot is shown in Table 1. That is, M u = 3.05 kg, m t = 0.2 kg, m c = 0.75 kg. The constant height of the CoM z c is 21.7 cm, and the height l u of the stick is 9 cm which is the length from the mass of the upper body to the CoM. The rotational inertia J is 0.026 kg/cm 2 , and the angular accelerationφ of the CoM is obtained by measurement of the gyroscope, which attached on the waist of TKU-X Robot.  In order to verify the performance of the proposed TLIPFM, the walking trajectory generators based on LIPM, DLIPM, and TLIPFM are established to achieve biped walking for TKU-X Robot. There are four force sensors in the each foot sole of TKU-X Robot to measure the ZMP response. Therefore, the performance of the walking trajectory generator, which is established by LIPM, DLIPM, and TLIPFM, can be obtained during biped walking through the obtained ZMP response.
In the walking trajectory generator, the natural ZMP trajectory is designed for TKU-X Robot to achieve biped walking by adding the double support phase and moving ZMP reference. In this experiment, some biped walking parameters used for TKU-X Robot are listed in Table 2. The ZMP responses of the walking trajectory generators based on the LIPM, DLIPM, and TLIPM during biped walking are shown in Fig. 14, Fig. 15, and Fig. 16, respectively. The dash-dot line is the natural ZMP trajectory generated by (10) and (11), while the solid line is the actual ZMP response during biped walking. The dashed lines are used to indicate the minimum and maximum boundaries of ZMP. The safety bound for the humanoid robot to walk stably is that all the ZMP response is within the minimum and maximum boundaries. These three experimental results illustrate that all three ZMP responses are within the safety bound. Furthermore, it can be seen that the ZMP response of the proposed TLIPFM is lower than the ZMP responses of the LIPM and DLIPM during the biped walking from 0 s to 8.96 s. Note that the ZMP response still swings from 8.96 s to 10.24 s after a completely test on the real small-sized humanoid. The ZMP response of the proposed TLIPFM is also still lower than the ZMP response of the LIPM and DLIPM. When calculating the error between the actual ZMP response and the natural ZMP trajectory in this experiment, the sum of absolute errors of these three models are listed in Table 3. This experiment can verify that the walking trajectory generator based on the proposed TLIPFM has a smaller ZMP error, which can make the robot walk more stably. In addition, the computational cost of the proposed TLIPFM is almost the same as that of LIPM and DLIPM because the rotational torque parameter C τ of the equation of motion and the natural period T TF of the TLIPFM VOLUME 8, 2020   can be pre-calculated as constants during the process of biped walking. The proposed TLIPFM method still retains the low computational cost characteristic of the conventional LIPM method. Therefore, in the case that the humanoid robot can stably walk without adding additional computational costs, the walking trajectory generator based on the proposed TLIPFM is superior to the other two methods.
By using the walking trajectory generator based on the proposed TLIPFM, the experimental results of the biped walking in sagittal and lateral plane are shown in Fig. 17 and Fig. 18, respectively (refer to supplementary video). In Fig. 17(a) and Fig. 18(a), TKU-X Robot is static standing in the starting position. From Fig. 17(b) and Fig. 18(b) to Fig. 17(e) and Fig. 18(e), the right foot is the support foot and the left foot moves forward. On the other hand, from Fig. 17(f)   FIGURE 17. Snapshot of the experiment of the biped walking in sagittal plane. and Fig. 18(f) to Fig. 17(i) and Fig. 18(i), the left foot is the support foot and the right foot moves forward. First, the CoM of TKU-X Robot swings from the middle to the right (left), as shown in Fig. 17(b),(f) and Fig. 18(b),(f). Next, the left (right) foot of TKU-X Robot lifts from the ground, as shown in Fig. 17(c),(g) and Fig. 18(c),(g). Then, the left (right) foot of TKU-X Robot moves one step forward, as shown in Fig. 17(d),(h) and Fig. 18(d),(h). Finally, the left (right) foot of TKU-X Robot steps to the ground, as shown in Fig. 17(e), (i) and Fig. 18(e), (i). The right foot will return to be the support foot and the left foot will move forward again. Note that the proposed method can also be used on other humanoid robots to quickly build their walking trajectory generators.

V. CONCLUSION
A dynamic model named TLIPFM is proposed for the humanoid robot and a natural walking trajectory generator based on TLIPFM is established to achieve stable biped walking. There are four main contributions of this research. First, in the construction of the proposed TLIPFM, three masses (the mass of the whole upper body of the robot, the thigh mass of the robot's support leg, and the calf mass of the robot's support leg) in the mass distribution of the humanoid robot is considered, and a flywheel joint is configured at the CoM of the robot to describe its rotational torque of the angular momentum. Compared with LIPM and DLIPM, the proposed TLIPFM considers both the three-mass distribution and the angular momentum of the robot, so it is closer to the walking motion of human. Second, a TLIPFM-based walking trajectory generator is established to generate ZMP trajectory, CoM trajectory, and foot trajectory in three-dimensional space so that the humanoid robot can walk stably. Moreover, the proposed TLIPFM and moving ZMP reference are applied to make walking trajectories more natural and human-like. Third, a laboratory-made small-sized humanoid robot is used to test the practicability of the proposed TLIPFM-based walking trajectory generator. Some experimental results are presented to verify the feasibility of the proposed method. By comparing the proposed TLIPFM with LIPM and DLIPM, it is proved that the proposed method indeed reduces the model error and let the robot can walk stably. Moreover, the proposed TLIPFM method still retains the low computational cost characteristic of the LIPM method, so it is more suitable for real-time biped walking of humanoid robots. Finally, this method is a general method for humanoid robots and can be easily and quickly applied to other humanoid robot platforms.