Virtual Worlds for Testing Robot Navigation: A Study on the Difficulty Level

The ability to navigate in diverse and previously unknown environments is a critical service of autonomous robots. We propose a test framework based on MORSE (Modular Open Robots Simulation Engine), and using the generation of virtual 3D worlds to challenge the navigation service. We elaborate on the notion of the difficulty of the generated worlds, which we characterize in terms of mission achievement, mission duration and trajectory curves. We experimentally study our ability to control the difficulty level by means of the generation parameters. We also assess the indeterminism of the navigation and how it evolves depending on the difficulty level. The experimental outcomes provide insights toward the definition of test strategies to further stress the navigation service.