Deep Reinforcement Learning-Based 3D Exploration with a Wall Climbing Robot

It is crucial to impart autonomy to robots for efficient exploration in surveillance and military applications. Most research on exploration deal largely with 2D environments using ground-based robots or 3D environments using drones. For applications requiring stealth, wall-climbing robots have an edge over drones however research is scant in 3D exploration algorithms using such robots. This paper presents a constrained 3D mapping problem and proposes a reinforcement learning-based exploration algorithm for the same using a lizard-inspired robot. The developed approach is based on a simulation-based framework in CoppeliaSim for simulating robot motion and Tensorflow for action selection algorithm using Deep Q Network (DQN). We report a number of simulation results indicating an improvement in median coverage by 20.3% using the developed DQN-based action selection over that of one generated by random action selection. Moreover, we observe that the developed DQN-based action selection approach led to more than 80% coverage within 150 steps, whereas the random action selection approach barely exceeded 60% coverage. We envisage that the developed approach can help imparting autonomy to the wall climbing stealthy robots in hostage scenarios.