Bio-Inspired Adaptive Locomotion Control System for Online Adaptation of a Walking Robot on Complex Terrains

Developing a controller that enables a walking robot to autonomously adapt its locomotion to navigate unknown complex terrains is difficult, and the methods developed to address this problem typically require robot kinematics with arduous parameter tuning or machine learning techniques that require several trials or repetitions. To overcome this limitation, in this paper, we present continuous, online, and self-adaptive locomotion control inspired by biological control systems, including neural control and hormone systems. The control approach integrates our existing modular neural locomotion control (MNLC) and a newly introduced artificial hormone mechanism (AHM). While the MNLC can generate various gaits through its modulatory input, the AHM, which replicates the endocrine system, adapts to rapid changes in online walking frequency and gait in response to different complex terrains. The control approach is evaluated on an insect-like hexapod robot with 18 degrees of freedom. We provide the results in three sections. First, we demonstrate the adaptability of the robot with the proposed artificial hormones. Second, we compare the performance of two robots with and without artificial hormones while walking on different complex terrains using three performance indexes (stability, harmony, and displacement). Third, we evaluate real-time online adaptations in the real world through real robot walking on different unknown terrains. The experimental results demonstrate that the robot with the proposed artificial hormones does not require several learning trials to adapt its locomotion. Instead, it can continuously adapt its locomotion online, thereby providing greater success and higher performance than other techniques when walking on all terrains.


I. INTRODUCTION
Walking robots can traverse different types of terrain very well, and their complex structure and control system drive this walking performance [1]. Many studies in the robotics field have aimed to apply various control systems to increase the walking performance on different terrains. For example, Bjelonic et al. [2], [3], Prágr et al. [4], and Zenker et al. [5] The associate editor coordinating the review of this manuscript and approving it for publication was Haiquan Zhao . used a vision sensor to visualize the type of terrain, and when the vision-based control system detected the terrain conditions, it triggered a change in the gait of the walking robot to achieve a lower cost of transportation (CoT). Fallon [6], Fućek et al. [7], and Kesper et al. [8] used a rotating LIDAR scanner and/or a stereo camera to detect obstacles and stairs to change the walking posture and foot placement of a walking robot accordingly. The aforementioned techniques rely on exteroceptive sensors to increase the robot walking performance [2]- [13]. This approach presents VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ challenges associated with not only the weight of the sensors and but also the related electronic components associated with these walking robots; greater weight corresponds to greater energy consumption. To overcome the limitation of the external sensor and added component weight, a proprioceptive sensor is instead considered. Focchi et al. [14] proposed a static-walking algorithm with proprioceptive feedback for a quadruped walking robot. The algorithm was derived from a dynamic model to adjust the robot foot placement. This technique applies proprioceptive sensors such as the ground reaction force and joint angle sensors to stabilize the robot body. However, many robot parameters must be predefined in this algorithm, and changes to any one parameter necessitate the manual recalculation of all the parameters. Studies have aimed to address the limitations in nonadaptive and predefined control parameters by developing a control system that can automatically adjust a robot's behavior under changes in the walking terrain. Arena et al. [15] introduced an error-based learning technique for a hexapod walking robot to learn motor skills for walking and climbing over obstacles.
Recently, Hwangbo et al. [16] proposed a reinforcementlearning-based method to learn the agile and dynamic motor skills of a walking robot to efficiently walk and recover from falling in complex configurations. However, these learning approaches [15], [16] typically require several trials or repetitions, and if the robot encounters an unknown situation, a new learning scenario is again needed, which is impractical and time consuming. Studies have investigated mechanisms to imitate the rapid, continuous, and online adaptation of living species for application in walking robots. One of the most effective mechanisms that has been implemented is the introduction of a hormone-based self-adaptive system in robots [17]- [22]. In nature, hormones are chemicals secreted by the endocrine glands that provide communication within the organism regarding disturbances from internal and external environments. Therefore, in certain cases, the hormone system is called an endocrine system. An artificial endocrine system and an artificial neural network have been introduced to improve power management and task allocation in robots [18], [19]. Sauzé and Neal [19] used this concept to maintain a steady-state and autonomous operation for an extended period in a sailing robot. Moioli et al. [20] applied a similar concept utilizing homeostatic influencing behaviors; their proposed multihormone evolutionary artificial homeostatic system was able to successfully coordinate and demonstrate differing reactions in wheel robots. Their system was also robust, managing both internal and external disturbances. Teerakittikul et al. [21] implemented an artificial hormone mechanism (AHM) in a wheeled robot to facilitate productive movements on uneven terrain and address possible faults occurring in the pitch sensory information of the robot. Although this hormone mechanism was not designed directly as a fault detection system in the robot, it could sense environmental cues and adjust the robot's behavior according to dynamic environments.
These previous works indicate that an AHM can provide fast, continuous, online adaptations; however, most research on AHMs applied to robots has focused on simple robotic systems (i.e., low-degree-of-freedom robotic systems) and simple movements. Therefore, an exciting advancement in the work proposed herein is the use of an AHM for a manydegree-of-freedom robotic system with complex movements to allow the robot to quickly and continuously adapt its walking behavior to unknown complex terrains without the several trials typically required during the learning process. This concept of online self-adaptive locomotion control forms the core of this work. Moreover, we describe how the hormone system, neural control, and proprioceptive sensory feedback can be combined for the adaptive locomotion of artificial walking systems, as observed in biological walking systems. Under this combination, the hormone system continuously regulates a neural control parameter, resulting in real-time walking frequency and gait adaptation. We validate the robustness of our control approach with six different unknown terrains (flat terrain, mesa terrain, ramp-up terrain, ramp-down terrain, rough terrain, and terraced terrain).
Taken together, the main contributions of this work are as follows: 1) A new online self-adaptive locomotion control technique is described for autonomously adaptive robot locomotion that allows for navigation on unknown complex terrains. 2) This technique, based on neural control and AHMs, does not need robot kinematics information, an environmental model, external (exteroceptive) sensors, or multiple learning trials or repetitions for adaptive robot locomotion.
3) The continuous real-time adaptation process of the control technique is performed online and relies on only a simple correlation of a predicted foot contact signal (obtained from a transformation of an efference copy or a motor command) and the actual incoming foot contact signal from each robot leg. Thus, this technique can potentially be applied to different walking robots with foot contact sensory feedback.

II. METHODS AND MATERIALS
This section explains the bio-inspired robotic platform used in this study, followed by a description of the online selfadaptive locomotion control system that allows the robot to perform various gaits and quickly adapt its gait in an online manner to deal with different unknown terrains. At the end of this section, we discuss how we tested and measured the system performance to demonstrate its efficiency and robustness.

A. BIO-INSPIRED ROBOT PLATFORM
To investigate and develop the proposed online self-adaptive locomotion control system, a hexapod robot was employed as our robotic system (Fig. 1).  [24] with maximum joint angles. Abbreviations: TL, CL and FL = Tx, Cx and Fx joints of the left leg; TR, CR and FR = Tx, Cx and Fx joints of the right leg; and 1, 2 and 3 = front, middle and hind legs, respectively.
A hexapod robot consists of six identical legs. Each leg consists of three joints that emulate the basic structure of a cockroach leg without a tarsus. These three joints are the thoraco-coxal (Tx) joint, the coxa-trochanteral (Cx) joint and the femur-tibia (Fx) joint. The maximum and minimum intervals of the joint movements of the legs are presented in Fig. 1(b) and (c), respectively. Each leg has a foot contact sensor. In total, the robot has 18 motors and six foot contact sensors. The foot contact sensors are used as inputs to our control system as described below.
The simulation toolkit LPZrobots [23], which creates a physically realistic simulation environment, was employed to first test our control scheme before finally evaluating the real robot ( Fig. 1(b)-(c)). This toolkit is based on the open dynamics engine (ODE). The physical parameters of the simulated robot are similar to those of the real robot in terms of the geometry, mass distribution, motor torque/speed, and sensors.
The robot was developed based on a modular robot framework (MORF) ( Fig. 1(b)-(c) [24]). Its 18 joints are driven by smart servo motors (4.2 N·m, XM430-350 Dynamixel). The foot contact sensory feedback of each leg is calculated from a motor torque sensor, built into the smart motor. The calculated foot contact feedback is used for the proposed controller. The robot is powered by a Li-ion battery (14.8 V,4 Ah). An Intel NUC i7 processor is used as the embedded controller. The robot has a weight of 4.5 kg.

B. ADAPTIVE LOCOMOTION CONTROL SYSTEM
The adaptive locomotion control system (ALCS, Fig. 2) is a bio-inspired mechanism that generates locomotion and enables real-time self-adaptation online. This ability is enabled by a combination of modular neural locomotion control (MNLC) and an AHM. While the MNLC can generate various gaits through its modulatory input (MI), the AHM, which replicates the endocrine system, allows for fast, continuous, and online walking frequency and gait adaptation. A schematic diagram of the ALCS is shown in Fig. 2. The proposed locomotion control system is inspired by a biological homeostatic control system [25]. In principle, the homeostatic control system can maintain balance in the body's internal state through an interaction between the nervous system (implemented as the MNLC in our control system) FIGURE 2. Adaptive locomotion control system (ALCS, in the frame) consisting of two main components: a modular neural locomotion control (MNLC, light gray) and an artificial hormone mechanism (AHM, dark gray). The system receives only foot contact feedback (proprioceptive feedback) and generates motor commands to control the joint angles of the robot legs. Parts of the motor commands are copied (efference copies) and transformed into expected foot contact feedback through a forward model in the AHM. The expected feedback is then correlated with the real foot contact feedback for the walking frequency and gait adaptation (see text for details). VOLUME 8, 2020 and the endocrine system (implemented as the AHM in our control system).

1) MODULAR NEURAL LOCOMOTION CONTROL
To generate various robot gaits, we use the MNLC (Fig. 3), which was developed by Manoonpong et al. [26]. The reason why MNLC is selected for our control system is because it is generic, does not require robot kinematics, and can generate various gaits by simply changing only one control input, i.e., the MI. This system consists of four key modules that are modeled using artificial neural networks. All neurons are standard additive nonspiking neurons. Although the functionalities of these neurons are not key contributors here, we briefly describe their functionalities because they are important for support of the artificial hormone system (described below) to generate adaptive locomotion. Further details of the neural models for each module are presented in [26]. The first module is a recurrent neural network with two tanh neurons and full connectivity. The synaptic connections of the network were empirically adjusted and set in accordance with the network dynamics, staying in the region of the Neimark-Sacker bifurcation, where the quasiperiodic attractors appear [27]. In this way, the network acts as a central pattern generator (CPG), generating rhythmic patterns with different frequencies. An extrinsic MI is projected to the mutual synaptic connections between the neurons to regulate the connections, thereby resulting in changing the frequency of the neural outputs or the CPG signals. When the MI is high, the walking frequency is high, leading to a fast gait, and when the MI is low, the walking frequency is low, leading to a slow gait, as illustrated in Fig. 4. In this study, the MI is regulated by the AHM for frequency and gait adaptation. The second module is a neural CPG postprocessing network that shapes the CPG signals from the first module to smooth the walking gait of the robot. The network is a simple feedforward network consisting of two binary threshold neurons and two integrator output units. The network parameters were manually tuned to obtain continuous and smooth ascending and descending motor control signals for smooth robot gait generation (see [26], [28] for more details).
The third module consists of two main neural networks to provide walking directional control of the robot. The first is the phase-switching network (PSN), which is a feedforward network. It has four hierarchical layers with 12 tanh neurons. The network parameters (such as the synaptic connections and bias terms) were manually designed (see [29] for more details). The network can switch the phase of motor control signals with respect to a given external input. This network is used for steering the robot to walk sideways. The second is the velocity regulating network (VRN), which is a feedforward network. It is formed by four hidden tanh neurons that are connected to one output tanh neuron. The network was trained by the backpropagation algorithm [29] to act as a multiplication operator. As a result, the network has the capability to increase or decrease the amplitude of the motor control signals and even reverse them with respect to external inputs. It can therefore be used to control different walking directions, such as forward/backward, turning left/right for different angles, or curve walking in the forward and backward directions (see [30] for more details).
The last module contains 18 motor neurons for controlling all leg joints of the robot. The motor control signals from the third module are transmitted to the motor neurons of the right hind leg and propagated sequentially forward through delay lines to the right middle leg, the right front leg, the left hind leg, the left middle leg and, finally, the left front leg to achieve different gaits. The final signal at each motor neuron is sent to the corresponding leg joint for position or joint angle control (see [28] for more details).

2) ARTIFICIAL HORMONE MECHANISM
To achieve fast, continuous, and online adaptation for maintaining balance in the robot's internal state, an AHM has been developed based on the one proposed by [21], [22]. The AHM is inspired by the endocrine system (also called the hormone system) in biology. In nature, the hormone system controls and regulates processes in the body to maintain homeostasis (i.e., the dynamic equilibrium of the body's internal state). Here, the AHM aims to maintain the dynamic equilibrium of the robot's internal state by releasing and regulating hormones, which influence the robot locomotion behavior. For our implementation, the AHM has three significant components, as indicated in Fig. 5(a): a hormone gland (HG), a hormone receptor (HR), and a target cell. The HG is primarily responsible for the secretion of hormones and regulating or controlling hormone production quantity (i.e., hormone concentration). The function of the HG is based on the hypothalamus and pituitary gland in a biological system. The hormone amount or concentration depends on stimuli from both internal and external environments and is subject to feedback control from other hormones. The HR is a mechanism used to determine the target cell where hormones can express their function and specify the influences on the target systems. The target cell is a mechanism of the controller, which can adjust the control parameter (in our case, the MI of the MNLC) to balance the body's internal state with the external state of the environment. The functions of the HR and the target cell are comparable to those of an insulin receptor in a biological system.
There are two main inputs connected to the HG: 1) the control input (CI) and 2) the signal input (SI). An input connected to the HG via the SI is used directly for the calculation of the level of hormone stimulation in each gland, while the CI provides a means to enable interactions among hormones and create hormone networks as outlined in (1): where release(t) is the hormone release function and CI (t) is the control input, which can be enabled (i.e., as a function of time) or disabled (i.e., at a constant value). If we enable this input, then there are four modes (inhibitory, stimulatory, negative feedback and positive feedback) to choose from to bias the hormones; see [22] for more details of these modes. For simplicity, here, we disable the input by setting it to one (i.e., CI (t) = 1.0). SI (t) is the signal input for stimulating the HG. It can be preprocessed through a signal input function SIF(SI (t)), which is the alternate form of the standard deviation formula, as shown in (2): where n is the number of legs (here, n = 6) and SI i (t) is the correlation between the normalized difference of the actual foot contact sensor signal and its mean value and the normalized difference of the corresponding predicted foot contact sensor signal and its mean value for leg i. The normalization is applied to compare the differences based on a value between zero and one. The correlation equation is described as (3): where SP i (t) is the actual foot contact sensor signal of leg i; FM i (t) is the predicted foot contact sensor signal of leg i, as calculated from (4); k is the period of the signals (in this work, 50 time steps); FM i (t) is the mean value of FM i over 50 time steps; SP i (t) is the mean value of SP i over 50 time steps; and σ SP i (t) and σ FM i (t) are the normal standard deviations of SP i (t) and FM i (t), respectively.
Here, FM i (t) is the forward model of leg i ( Fig. 6(b)), µ is the scaling factor used to adjust the signal to the same range of foot contact signal, w is the low-pass filter scale and motor i is the motor command of the Cx joint (see Fig. 1). In (4), we set µ and w to 0.3 and 0.5, respectively. In principle, the model translates a motor command (i.e., an efference copy, Fig. 6(a)) into a suitable predicted foot sensor signal to correlate with the actual incoming foot contact signal. Here, the actual incoming foot contact signal is an analog signal varying from zero to one, where zero means the foot does not touch the ground and one means the foot fully touches the ground.
In the final action of HG, to provide the hormone concentration at each time step (Cg(t)), the hormone concentration from the previous time step is combined with the level of hormone stimulation from release(t), as shown in (5): where C g (t) is the time-dependent hormone concentration, t −1 indicates the previous time, α is the stimulation rate, β is the decay rate, and release(t) is the hormone release function (see (1)).
In this work, α and β are empirically set to 0.29 and 0.982, respectively, from (5). We combine the AHM and the MNLC, resulting in the ALCS, by setting the HR parameter of the AHM to the MI parameter of the MNLC. The HR is driven by the hormone concentration (C g (t)) as described by (6), while the MI of the MNLC is continuously regulated through (7).
Here, MI_H represents the MI of the CPG network in the MNLC regulated by the HR. Based on the AHM and MNLC mechanisms of the proposed ALCS, we can adapt the walking frequency and gait of the robot continuously online. The AHM aims to maintain the balance of the hormone concentration (C g (t)). If the concentration increases beyond a defined middle point (here, 0.5), the MI value increases, resulting in a faster walking frequency and gait. However, if the concentration of the hormone decreases below the defined point, the MI value decreases, resulting in a slower walking frequency and gait.

C. EXPERIMENTAL DESIGN
We aim to demonstrate the performance of the ALCS with online adaptation for robot walking. Therefore, the experiment is designed to support investigations from three perspectives. First, the ability of the ALCS to assist the robot in reaching an equilibrium condition is evaluated. Second, the robot performance as controlled by the MNLC (i.e., without online adaptation) and the ALCS (i.e., with online adaptation from the AHM) is compared. Third, real-time online adaptations in the real world through real robot walking on different unknown terrains are evaluated. The three main experiments were performed as detailed below.

1) EXPERIMENT 1: ADAPTABILITY OF THE ALCS-DRIVEN ROBOT ON CHALLENGING TERRAINS
In this experiment, the MI was initially set to different values of 0.02, 0.04, 0.06, 0.09, 0.12 and 0.19. The ability of the ALCS-driven robot to walk on six types of terrain was tested: (a) flat terrain, (b) mesa terrain, (c) ramp-up terrain, (d) ramp-down terrain, (e) rough terrain, and (f) terraced terrain. The profile of each type of terrain is shown in Fig. 7.

2) EXPERIMENT 2: COMPARISON OF THE WALKING PERFORMANCE BETWEEN THE ALCS AND THE MNLC ON DIFFERENT TERRAIN TYPES
In this experiment, the ALCS-and MNLC-driven robots were tested in a task involving walking from the starting point to the endpoint on each type of terrain indicated in Fig. 7. The experiment was repeated 16 times for each terrain. The MI value was initially set to 0.19 for the ALCS-and MNLC-driven robots because this setting leads to a typical fast tripod gait (Fig. 4). The walking performance was measured with the performance index, which is described in the next section.

3) EXPERIMENT 3: VALIDATION OF THE WALKING PERFORMANCE OF THE ROBOT IN THE REAL WORLD
This experiment was intended to demonstrate the online self-adaptation ability of a hexapod robot driven by the ALCS. We let the robot walk on four different unknown terrains, namely, 1) ramp-up terrain, 2) ramp-down terrain, 3) terraced terrain with two different steps and 4) natural uneven terrain, as shown in Fig. 8. We also compared the performance of the ALCS with the MNLC and the MNLC with searching and elevator reflexes [26].

D. PERFORMANCE INDEX
To evaluate the walking performance, three performance indexes were chosen to measure the stability, harmony and displacement. The stability indicates the possibility that the robot will fall and decelerate. The harmony indicates conformation of the robot body position with the change in the terrain level [31]. The displacement is a measure of three perspectives: 1) excessive traveled distance, 2) lateral displacement, and 3) the ratio between the actual distance on the X-axis and the calculated distance with the maximum velocity.
According to Ferreira and Santos [32], the determined values of these three indicators should generally be between zero and one. We assume here that a value of one corresponds to the best performance for these indicators.

1) STABILITY
For the stability, we use the acceleration (Acc) and the angular velocity (Ang) of all three axes from the simulation to determine the Acc minimum peak value (Acc min ) and the Ang peak-to-peak value (Ang PP ) of each axis using (8) and (9), respectively, as shown in Fig. 9. Ferreira and Santos [32] defined the minimum value of Acc min as A min and the maximum value of Ang PP as PP max for all three axes. The maximum values of both A min and PP max are used to determine robot stability. The robot stability criterion was calculated using (10), which is an average of the exponential decays of A min and PP max . In this study, λ is the decay constant, e.g., 0.1. Thus, the maximum stability value is 1.0, meaning that the robot walks without any body oscillations. The lower the stability value is, the greater the robot body oscillations, VOLUME 8, 2020 which can lead to inefficient walking.

2) HARMONY
The harmony is derived using (8) and (9), where RI A min and RI PP min are calculated using (11) and (12) shown below and RI A and RI PP represent the highest values selected from the three axes. The harmony was calculated from RI A and RI PP , as shown in (13). Fig. 10 shows the behavior of a signal to explain the values of the stability and harmony. In the worst case, the signal has a high magnitude and diverges (see Fig. 10(a)). In the best case, the signal has a low magnitude and very low divergence (see Fig. 10(d)).

3) DISPLACEMENT
The displacement was calculated using (14), as indicated below: where x is the distance on the X-axis from the starting point to the finishing point, s is the distance traveled from the starting point to the finishing point, r is the total displacement calculated, T exp is the experiment time, and V max is the robot's maximum velocity. | x| s is the excessive traveled distance. If the excessive traveled distance equals one, then the robot walks forward in a straight trajectory with no deviation. | x| r is the lateral displacement. If there is no deviation from the X-axis when the robot reaches the destination, then the lateral displacement is one. The last part is the ratio between the actual distance on the X-axis and the calculated distance with the maximum velocity. For example, the orange line represents the best-case scenario in which the robot walks straight towards the trajectory; as a result, the traveled distance and lateral displacement value are one. The blue line indicates the second scenario, when the robot does not walk toward the trajectory right away, although it eventually reaches the target point. In this case, the distanc traveled is less than one, while the lateral displacement remains at one, as shown in Fig. 11.

III. RESULTS OF THE ONLINE ADAPTABILITY OF THE ALCS-DRIVEN ROBOT ON CHALLENGING TERRAINS
The initial values of MI_H are set to 0.02, 0.04, 0.06, 0.09, 0.12 and 0.19 (i.e., different gaits) for all terrain types. The MI_H values converge to a specific value between 0.047 and 0.053 while walking on flat terrain, as shown in Fig. 12(a) (i.e., tetrapod gait). The MI_H of the mesa terrain, the ramp-up terrain, and the ramp-down terrain converge to 0.048-0.059, as indicated in Fig. 12(b)-(d), meaning one definite robot gait was achieved (in this case, the tetrapod gait). In the last period of walking on the mesa terrain, MI_H increased to approximately 0.065 because the robot is approaching the transition section between the down slope and the level ground of the mesa terrain, becoming slightly stuck. Therefore, the controller adapts the locomotion to attempt to prevent the robot from becoming stuck.
For the rough and terraced terrains, MI_H varies between 0.048 − 0.15, as shown in Fig. 12(e)-(f). This MI_H range for the rough and terraced terrains covers four gaits: the tetrapod gait, the caterpillar gait, the intermixed gait, and the tripod gait. For the rough terrain, MI_H shows unpredictable patterns, and there is no correlation with changes in the terrain level because the terrain had a nonuniform pattern. In contrast, for the terraced terrain, MI_H shows predictable repetitive patterns (resulting in tetrapod, intermixed, caterpillar and tripod gaits) that changed with changes in the terrain level. In Fig. 12, MI_H exhibits a pattern that is indirectly characteristic of the terrain upon which the ALCS-driven robot walked.

IV. RESULTS OF THE COMPARISON OF THE WALKING CAPABILITY ON THE SIX TERRAIN TYPES
From the 16 repeated experiments, the successful runs were counted and considered in the analysis. The robots were considered successful when they reached their target or destination within a certain time period, e.g., 20 minutes. With the time limit, for the flat terrain, the robots must reach a certain distance. For the mesa terrain, the robots must walk up and down the slopes. For the ramp-up/ramp-down terrain, the robots must walk up/down to the end of the ramp. For the rough terrain, the robots must walk across the terrain from one end to the other end. For the terraced terrain, the robots must walk up and down stairs. Fig. 13 shows a comparison of successful runs between the ALCS-and MNLC-driven robots for each terrain type. The ALCS-driven robot can manage to locomote on every terrain, while the MNLC-driven robot fails to deal with all terrains. For the random rough terrain (T5), which is the most difficult type, the ALCS-driven robot succeeded 10 out of 16 times, while the MNLC-driven robot succeeded three out of 16 times and only by chance. Successful runs for each robot type on each terrain type. The blue bars represent the ALCS's successful runs, and the orange bars represent the MNLC's successful runs. T1-T6 represent each terrain type as follows: T1: flat terrain, T2: mesa terrain, T3: ramp-up terrain, T4: ramp-down terrain, T5: rough terrain, and T6: terraced terrain (see Fig. 7).

A. STABILITY AND HARMONY ANALYSIS
The stability is a measure of the possibility that the robot will fall and decelerate. Fig. 14(a) reveals that the stability of the ALCS is higher than that of the MNLC based on the ability of the ALCS-driven robot to adjust its foot placement according to changes in MI_H.
The harmony measures the robot's body swing when it walks on different types of terrain. Fig. 14(b) reveals that the ALCS exhibits better harmony than the MNLC. When a robot elevates its foot and begins walking, it automatically loses its body balance. The ability of the ALCS-driven robot to adjust MI_H assists in varying the foot placement to maintain the robot's body balance. As a result, the body swing rate is reduced.
When we evaluate the stability and harmony results of the ALCS-driven robot from all six terrains, the results indicate a better walking performance for the ALCS-driven robot than for the MNLC-driven robot. The stability and harmony results were correlated, showing the same trends for all terrain types except for the terraced terrain.
In the terraced terrain, the stability is low, while the harmony is high. This contradiction can be explained as follows. When the ALCS-driven robot climbs the steps, it must learn to climb. Additionally, when it hits the step, its deceleration increases, resulting in lower stability. In contrast, the harmony result is high because the robot adjusts its swing with changes in the terrain level. As shown in Fig. 12(f), MI_H varies between 0.048 and 0.15, which falls into the range of the caterpillar gait (0.07-0.1). With this gait, the ALCS-driven robot can use its two front legs to climb the first step with support from the other two pairs of legs. When it reaches one step, MI_H is maintained because all six legs have not touched the ground. As a result, it walks up the following step more quickly, resulting in a lower swing value.

B. DISPLACEMENT
In this section, we compare the displacement results between the ALCS-and MNLC-driven robots for each type of terrain. Fig. 14(c) shows that the ALCS displacement is greater than that of the MNLC because the ALCS-driven robot can walk in a more straightforward manner along the trajectory. For the MNLC-driven robot, because of its tripod gait, all three legs are elevated at the same time, resulting in less friction and the ability to achieve a higher speed. When this occurs, however, the robot may slip and deviate from the target destination. Notably, in the beginning of the test, neither system walks straight to the target destination because they were initialized with a specific fast gait (i.e., the tripod gait). Nevertheless, the ALCS better develops a balanced position due to the AHM, which reduces the walking frequency by lowering MI_H; thus, the deviation from the trajectory is reduced. As a result, the traveled distance and the lateral displacement are closer to one than they would be otherwise, as shown in Fig. 15, meaning that the robot walked nearly in a straight line. Notwithstanding, the estimate of the traveled distance from the calculation of the maximum velocity and the travel time of the ALCS-driven robot is less than that of the MNLC-driven robot simply because the ALCS-driven robot walks more slowly (but stably).

FIGURE 15.
Walking trajectories of the MNLC-driven and ALCS-driven robots on flat terrain. Since the AHM of the ALCS can adapt the robot gait online during walking, the robot can use an appropriate gait that balances its body posture, which results in straight walking.

2) WALKING ON MESA TERRAIN
The profile of the ramp used in these experiments is split into three parts, as depicted in Fig. 7(b). The first part is a ramp up with an inclination of 7.5 degrees. The second part is a flat terrain. The final part is a ramp down at a similar angle to the first part. Under this testing scenario, the MNLC-driven robot fails to achieve excellent walking performance. It climbs up the slope with a tripod gait, which yields minimal contact surface and results in a deviation in its trajectory path.
In contrast, the ALCS utilizes the AHM to automatically adjust its gait to a tetrapod gait through MI_H, resulting in more contact with the ground surface. A comparison of the velocities when climbing up and climbing down is shown in Fig. 16(b); the latter velocity is higher despite the similar MI_H values, although obvious effects of gravity were observed. Fig. 16(c) shows various MI_H values when the ALCS-driven robot walks on changing terrains, indicating the effort needed to adjust the walking gaits to cope with the different types of walking environments.
An experiment conducted on the ramp-up terrain in Fig. 7(c) yields a result similar to that of the mesa terrain in state 2. Similarly, the result of the ramp-down terrain in Fig. 7(d) is approximately the same as that of the mesa result in state 4, as shown in Fig. 16.

3) WALKING ON ROUGH TERRAIN
The rough terrain involved many curves with random nonuniform distributions, as shown in Fig. 17. Compared with the other terrains, this terrain was likely the most challenging due to the continuous disturbances and lack of predictability. The MNLC-driven robot fails to reach the target destination due to its constant MI (fixed gait) when it becomes stuck on the surface. In contrast, the ALCS-driven robot shows a better displacement result than the MNLC-driven robot because its ability to adjust MI_H helped it address the changing terrain curve. Therefore, the ALCS-driven robot does not become stuck on the rough surface, although the robot still deviated from the trajectory. More specifically, its MI_H adjustment continuously supported the change in relative phases between the legs, resulting in gait changes when dealing with terrain curve changes.

4) WALKING ON TERRACED TERRAIN
The terraced terrain consists of four steps (each 3.5 cm in height), as shown in Fig. 7(f). In this test, the MNLC-driven robot fails to reach the target destination for two main reasons: 1) while the MNLC-driven robot can navigate the first step, it sometimes hits the corners of the following steps and becomes stuck there, and 2) when the MNLC-driven robot hits a corner, it can no longer climb up the following steps. In contrast, the ALCS-driven robot exhibits an MI_H adjustment when there is a change in the terrain level, similar to the adjustment observed for the rough terrain. This analysis is divided into two sections to consider climbing up and climbing down.
The first part analyzes the climbing-up behavior, as shown in Fig. 18. S1-S4 are the periods when the ALCS-driven robot climbs up for each step, illustrating that the Cg value converges to oscillate around the middle point (i.e., 0.5). This action allows the robot to perform slightly different gaits to deal with the terrain. The second part refers to the climbing-down behavior (S5-S8) and indicates an enormous swing in the Cg value from the middle point, which results in a rapid change in the MI_H value (Fig. 18(c)). With this change, the ALCS-driven robot switched its gait between a fast gait (caterpillar gait) and a slow gait (wave gait). This behavior occurs when the ALCS-driven robot is climbing down, and the indirect influence of gravity accelerates the motion of the robot. Therefore, the Cg value increases, and if the motion of the robot moves too fast, the Cg value starts to decrease to slow down the walking speed and keep the robot stable. Notably, this rapid gait switching behavior was not predefined and emerged from the correlation of the predicted foot contact signal and the actual incoming signal that adjusts the hormone concentration of the AHM (adaptation process). This represents an affinity for living creature behavior [33]. From the analysis of the climb-up and climb-down behavior, the ALCS-driven robot has better climbing ability than the MNLC-driven robot because the Cg of the AHM helps to adapt the walking performance to changes in the terrain.

V. RESULTS OF A REAL ROBOT WALKING ON DIFFERENT UNKNOWN TERRAINS
We test three different controllers on the real robot: the ALCS, the MNLC, and the MNLC with conventional searching and elevator reflexes. 1 The three indexes (i.e., stability, harmony and displacement) are used for our evaluation. A typical tripod gait is used as the initial gait for all controllers.
The results show that the ALCS allows the robot to adapt its gait online to successfully and stably walk on different terrains, including 1) ramp-up terrain, 2) ramp-down terrain, 3) terraced terrain with two different steps and 4) natural uneven terrain (Fig. 8). Fig. 19(a) indicates that the ALCS-driven robot shows the highest stability in all terrains. This is because the ALCS adapts MI_H online to obtain slower and more stable gaits to deal with the change in terrain, while the MNLC and the MNLC with the reflexes do not have a mechanism for gait adaptation. In those cases, the robot maintained the tripod gait. In general, walking with the tripod gait on uneven terrain can easily lead to instability. However, reflexes can help extend or elevate the legs when missing the ground contact during a stance phase or stepping on or hitting an obstacle during a swing phase. Therefore, the MNLC with reflexes shows better results than the pure MNLC. For the harmony ( Fig. 19(b)), the MNLC with the reflexes showed the worst result in all terrains. This is because the searching and elevator reflexes made the robot body move up and down a lot (i.e., body swing at a high magnitude). For example, when the robot greatly extends the leg to obtain ground contact (searching reflex), the body height is increased (moving up), FIGURE 19. Charts of the stability (a), harmony (b) and displacement (c) results of the ALCS-driven robot, the MNLC-driven robot, and the MNLC-driven robot with reflexes during walking on the ramp-up, ramp-down, terraced, and natural uneven terrains (see Fig. 8). and when it greatly elevates the leg to avoid hitting the terrain (elevator reflex), the body height is decreased (moving down). For the displacement (Fig. 19(c)), the ALCS shows the best results in all cases except walking on the ramp-down terrain, where its displacement is less than that of the MNLC with reflexes. However, the MNLC with reflexes showed the least displacement for the natural terrain (i.e., uneven terrain) because, from extending and elevating individual legs independently, the left and right legs might not have an equal distribution of ground contact points on average; therefore, the robot could easily deviate from a straight path. Fig. 20 reveals that walking on the ramp-up terrain consists of five states. The first, third, and final states are on level ground. As a result, MI_H converges to values between 0.027 and 0.037 ( Fig. 20(a)), which leads to a wave gait. The second and fourth states are the transitions from the level ground to the ramp and the ramp to the level ground, respectively. As a result, MI_H converges to values between 0.025 and 0.045 ( Fig. 20(a)), which leads to a mixture of wave and tetrapod gaits. The strong variation in MI_H (states 2 and 4) is because the expected foot contact sensor feedback derived from the motor commands and the actual foot contact feedback are not highly correlated. The supplementary video 1 of this experiment can be found at http://www.manoonpong.com/IEEEACCESS/Supplevideo1. mov. Fig. 21 reveals that walking on the ramp-down terrain consists of five states. All states are the same as for the ramp-up terrain. Therefore, the ALCS-driven robot exhibited similar walking behavior as that observed in the ramp-up terrain. The supplementary video 2 of this experiment can be found at http://www.manoonpong.com/IEEEACCESS/Supplevideo2. mov. Fig. 22 reveals that walking on the terraced terrain consists of five states. The first, third, and final states correspond to level ground. As a result, MI_H converges to values between 0.0283 and 0.042 (Fig. 22(a)), which leads to a wave gait.  The second and fourth states are the transitions at the edge. As a result, MI_H swings to values between 0.026 and 0.164 ( Fig. 22(a)), which leads to a mixture of wave, tetrapod, caterpillar, and tripod gaits. The strong variation in MI_H (states 2 and 4) is because the expected foot contact sensor feedback derived from the motor commands and the actual foot contact feedback are not highly correlated. The supplementary video 3 of this experiment can be found at http://www.manoonpong.com/IEEEACCESS/Supplevideo3. mov. Fig. 23 reveals that walking on the terraced terrain consists of five states. The first state corresponds to the cement road. As a result, MI_H converges to values between 0.026 and 0.038 ( Fig. 23(a)), which leads to a wave gait. The second state is the transition to grass. As a result, MI_H swings to values between 0.026 and 0.066 ( Fig. 23(a)), which leads to a mixture of wave and tetrapod gaits. The third and fifth states corresponded to grass. As a result, MI_H swings to values between 0.03 and 0.055 ( Fig. 23(a)), which leads to a tetrapod gait. The fourth state represents the ALCS-driven robot becoming stuck on the grass. As a result, MI_H swings to values between 0.03 and 0.084 ( Fig. 23(a)), which leads to a mixture of wave, tetrapod, and caterpillar gaits. The strong variation in MI_H (state 4) is because the expected foot contact sensor feedback derived from motor commands and the actual foot contact feedback are not highly correlated. The supplementary video 4 of this experiment can be found at http://www.manoonpong.com/IEEEACCESS/Supplevideo4. mov.
Altogether, the results of these experiments indicate that the behaviors of the ALCS-driven robot are consistent with the simulation, thus confirming the continuous real-time adaptation of the ALCS.

VI. CONCLUSIONS AND FUTURE WORK
The experimental results indicate that the ALCS-driven robot exhibited the greatest adaptability of the robots tested under different challenging terrains. The displacement, stability, and harmony results were used to determine the robot's walking efficiency. The frequency and gait of the robot were automatically adapted according to the changes in terrain. This adaptability results from the correlation of the predicted foot contact signal (obtained from a forward model) and the actual incoming foot contact signal of each leg. The correlation is used as a signal input for stimulating the HG to release the hormone. The hormone then binds with its receptor (HR) on the target cell so that the MI of the MNLC is modulated online, and the gait pattern of the robot is also adjusted in real time.
Many efforts have been dedicated to achievie online gait adaptation of legged robots. In previous studies, a robot's stable position was analyzed to apply the kinematics of its step to calculate optimized joint angles for appropriate foot placement [34], [35] and path planning [36]. A similar study was conducted by Focchi et al. [14], who applied a torque-controlled technique to determine foot placement and maintain a robot's balance on a high-slope terrain.
In addition to this approach, some studies utilized exteroceptive sensing for terrain prediction and gait adaptation. Murata et al. [9] used a depth camera to recognize the terrain. The terrain information was then applied to a foot placement method to automatically adjust the body posture of a hexapod robot and the swing and stance movement of each leg of the robot. However, in the method, the foot placement position in each step was assigned by the operator (i.e., human in the loop) in order to select the next appropriate contact point for stable walking. Bjelonic et al. [2], [3] used stereo vision and IMU sensors to predict properties of the terrain (e.g., roughness and step height). This information was served to adapt several locomotion control parameters (e.g., step frequency and step height) of each leg of a hexapod robot. Another widely used approach is to utilize the learning capability of a robot to cope with different types of terrain [15].
Compared to these state-of-the-art techniques, one of the advantages of the ALCS is that an ALCS-driven robot can adjust its frequency online to maintain its stability without any foot placement calculation, exteroceptive sensors, or even prelearning of the changing environment. This continuous online frequency adaptation helps determine terrain characteristics indirectly, which is shown by the results of the adaptability of the ALCS-driven robot on challenging terrains (Fig. 12).
Another advantage of the ALCS-driven robot is that the combination of the AHM and the MNLC represents a simple but effective method for achieving adaptive locomotion without the need for complex sensory systems (such as vision or an exteroceptive sensor as often used by others [2]- [13]). This feature aids the development of a lightweight robot with minimal data and less time required for complex sensory signal processing and complex decision-making mechanisms.
Compared with our previous study on artificial hormonebased adaptive control for leg damage compensation [37], in this study, we show gait adaptation in response to different complex terrains. Furthermore, we demonstrate that the type of HR required to stimulate the target cell is different; thus, we introduce a previous value of MI_H (as shown in (7)), which results in more stable gait adaptation under different terrains. The mechanism of the hormone in our previous study without the previous MI_H value caused excessively rapid gait changes, leading to unstable adaptation.
This research indicates that the involvement of the AHM helps improve the ability of the robot to handle complex unknown situations. These findings support the assumption that the AHM allows the robot to achieve better walking performance without having to prelearn the unknown environment. Such a scenario is similar to the process in living organisms, where hormones influence neuromodulators to regulate motor control (i.e., CPG). Kravitz and his team [38], [39] injected serotonin (monoamine neurotransmitter) into lobster, which affected motor control (posture and escape) because serotonin induces the neuromodulator at the stomatogastric ganglion (STG). The STG seems to act as a mechanism involving neuromodulation and hormones that assists in controlling motor control for movement generation. In arthropods, Knebel et al. [40], [41] have demonstrated that the subesophageal ganglion (SEG), which connects the brain and thoracic ganglion, affects motor control for movement generation [42]. Neurobiologists have hypothesized that neuromodulation and hormones seem to be integrated in the SEG (personal communication). Additionally, many reports have discussed neuromodulation in relation to a set of motor patterns that are very carefully orchestrated and controlled by different peptides acting as neurohormones or neuromodulators [43]- [45]. Our study also supports these findings and proposes an option for how hormones interact with motor control though neuromodulation (i.e., MI_H) in an artificial walking system, as illustrated in Fig. 2.
Future work will aim to extend this study by developing an AHM mechanism that enables the robot to control its walking direction for autonomous goal-directed navigation. Slip compensation was not considered in the ALCS proposed herein. In future work, we plan to extend our hormone system to a hormone network with other proprioceptive feedbacks to address this problem.