Energy-Efficient Gait Optimization of Snake-Like Modular Robots by Using Multiobjective Reinforcement Learning and a Fuzzy Inference System

Snake-like modular robots (MRs) are highly flexible, but, to traverse a challenging terrain or explore a region of interest, MR needs to attain efficient locomotion depending on a tradeoff between objectives like forward velocity and power consumption of the robot. The objectives can vary with different weights depending upon the situation, reflecting relative objective importance. This study developed a multiobjective reinforcement learning algorithm based on a fuzzy inference system (FI-MORL) to select the most appropriate gait parameters of snake-like MRs according to the objective weights. The developed algorithm employs a fuzzy inference system to reduce the number of states in an environment, which results in faster learning. The proposed approach uses the previously learned experience to rapidly achieve the best objective values in response to a change in weights. While setting equal importance to the objectives, FI-MORL delivers superior performance than single-objective reinforcement learning algorithms by consuming 2% less power and gaining 2.5% higher velocity since it mitigates the effect of weight change, similar performance found comparing an actor-critic algorithm. Likewise, the proposed method outperforms by consuming 14% less power and achieving 11% higher velocity than traditional methods like proximal policy optimization, deep Q-network, and vanilla policy gradient. Even after weight change, FI-MORL achieved a 14% higher reward than the above methods. The proposed FI-MORL framework can effectively converge quicker and efficiently handle the changes in objective weights.