A Framework for Generating Synthetic Expert Demonstrations in Digital Twin-based Robot Learning

We present a simulation-assisted imitation learning framework for robot pick-and-place tasks. A digital twin of a Techman TM5-900 robot arm with an OnRobot RG6 gripper is deployed within NVIDIA Isaac Sim with ROS 2 integrated control. The system enables efficient collection of expert demonstration trajectories in a realistic simulated environment, addressing the data efficiency and safety challenges of imitation learning. One of the three deployed ROS 2 nodes manages the virtual environment and task flow through API calls from the Omniverse Python packages. A solver module generates demonstrations on generalized labautomation pick-and-place tasks, which are executed in simulation and recorded by a recorder node. The integration takes advantage of Isaac Sim’s high-fidelity physics and ROS 2 interface (ROS 2 Bridge and Action Graphs) to ensure that the simulated robot behavior can be transferred to real hardware. The results indicate that our framework can produce reliable demonstrations on a scale, laying the groundwork for effective training of robot manipulation policies via imitation learning.