A Real-Time Simulator for High-Speed Structured-Light Cameras in Robotic Applications with Optimal Positioning Evaluation

With recent advances in industrial robotics, autonomous robotic arms need to deal with complex environments using sensors such as depth cameras. Structured-light cameras are often employed for this purpose as they provide high precision measurements and increasing speed acquisition. Furthermore, as digital twins are becoming more common for training and guiding real robots, there is a growing need for high-performance sensor simulations. This paper thus introduces a fast, GPU-implemented, and realistic simulation tool for structured-light depth cameras. It can compute 3D point clouds with noisy coordinates in real-time for a set of cameras in complex, dynamic scenes simulated by a physics engine. It can also interactively evaluate coverage and overlap of simulated point clouds on objects of interest. Finally, by maximizing their values, as a pre-process or during simulation for motionless scenes, it allows optimal cameras positioning. To demonstrate the capabilities of our simulation tool, we present two use cases. The first showcases a real-time simulation of a high-speed structured-light camera operating at 20 frames per second, mounted on a robotic arm that dismantles an electrical battery. The second illustrates the optimal positioning of a set of fixed structured-light cameras in a cluttered environment, where a robotic arm performs the cutting of a glove box. To our knowledge, our tool is the first to offer such real-time simulations and rapid structured-light camera positioning in complex robotic environments.